-- Logs begin at Wed 2022-08-24 06:03:00 UTC, end at Wed 2022-08-24 07:03:01 UTC. -- Aug 24 06:28:58.955662 dv-cinder-854271-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 06:28:58.965883 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 06:28:58 2022] *** Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: nodename: dv-cinder-854271-1-iscsi Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: machine: x86_64 Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: clock source: unix Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: pcre jit disabled Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: detected number of CPU cores: 4 Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: current working directory: / Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: detected binary path: /usr/bin/uwsgi-core Aug 24 06:28:58.968263 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: your processes number limit is 47680 Aug 24 06:28:58.968990 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: your memory page size is 4096 bytes Aug 24 06:28:58.968990 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: detected max file descriptor number: 2048 Aug 24 06:28:58.968990 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: lock engine: pthread robust mutexes Aug 24 06:28:58.968990 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: thunder lock: enabled Aug 24 06:28:58.968990 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 06:28:58.968990 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 06:28:59.011862 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: Python main interpreter initialized at 0x5639bccac9c0 Aug 24 06:28:59.011862 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: python threads support enabled Aug 24 06:28:59.011862 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: your server socket listen backlog is limited to 100 connections Aug 24 06:28:59.011862 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: your mercy for graceful operations on workers is 90 seconds Aug 24 06:28:59.012230 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 06:28:59.012230 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: *** Operational MODE: preforking *** Aug 24 06:28:59.012534 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: *** uWSGI is running in multiple interpreter mode *** Aug 24 06:28:59.012534 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: spawned uWSGI master process (pid: 81757) Aug 24 06:28:59.012732 dv-cinder-854271-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 06:28:59.013954 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: spawned uWSGI worker 1 (pid: 81768, cores: 1) Aug 24 06:28:59.013954 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: spawned uWSGI worker 2 (pid: 81769, cores: 1) Aug 24 06:28:59.013954 dv-cinder-854271-1-iscsi devstack@keystone.service[81757]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 06:29:03.279562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:29:03.279562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: from cryptography.utils import int_from_bytes Aug 24 06:29:03.280436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:29:03.280436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: from cryptography.utils import int_from_bytes Aug 24 06:29:03.281847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:29:03.281847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: from cryptography.utils import int_from_bytes Aug 24 06:29:03.282339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:29:03.282339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: from cryptography.utils import int_from_bytes Aug 24 06:29:03.335740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 06:29:03.335984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 06:29:03.336160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] command line args: [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 06:29:03.336381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 06:29:03.336589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ================================================================================ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 06:29:03.336843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] admin_token = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.337048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config_dir = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.337270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.337421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config_source = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.337627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] debug = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.337828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.338135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 06:29:03.338225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 06:29:03.339293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] command line args: [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 06:29:03.339443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 06:29:03.339582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ================================================================================ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 06:29:03.339713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_config_append = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] admin_token = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.339973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config_dir = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_dir = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config_source = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] debug = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.340942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_config_append = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.341894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_dir = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.342929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.343927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] publish_errors = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.344963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] standard_threads = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.345945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_journal = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_json = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] publish_errors = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_stderr = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_syslog = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.346880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.347020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.347103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.347177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.347384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.347481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] standard_threads = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.347621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.347719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.347874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.347959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.348086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.348168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.348247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_journal = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.348329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.348404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_json = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.348564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.348648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_stderr = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.348751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_syslog = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.348840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:29:03.349085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.349975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.350928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351527 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.351926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.352971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.353912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354067 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.354843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.355973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.356933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.357129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.357212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.357423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.357492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.357681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.357766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.357930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.358971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.359929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.360895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.361948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.362964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.363943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.364892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.365978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.366927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.367990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.368928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.369993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.370951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.external = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.371995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.372146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.372225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.372432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.372574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.372681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.372903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.password = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.372989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373136 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.token = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.373944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.external = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.374873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.password = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.token = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.375877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.376993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.377966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.378940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.379988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.380946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.381920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.382867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.383846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.384935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.385977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.386923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.387880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.388202 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.388332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.388464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.388727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.388810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.388971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.389213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.389450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.389691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.389925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.390164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.390410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.390489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.390653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.390742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.390907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.390987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.391974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.392911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.393927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.394981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.395944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.396890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.397861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.398932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.399919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.400900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.401993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.402069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.402227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.402320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.402476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.402662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.402742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.402930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.403114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.403249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.403406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.403610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.403871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.403997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.404097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.404340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.404421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.404573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.404808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.404888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.405954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.406925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.407979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.408947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.409899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.410859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.411930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.412993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.413157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.413361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.413442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.413608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.413688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.413849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.413929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.414882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.415995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.416961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.417984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.418912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.419950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.420997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.421935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.422878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.423853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.424882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 06:29:03.424959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.425191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.425444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.425685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.425933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.426170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.426417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.426655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:29:03.426829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 06:29:03.454634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.454858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.455083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.456426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.456664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.456895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.463482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.465357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.491986 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.492129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.493712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.493965 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.544950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.545898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.598574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.598778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.598848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.599053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.758816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759697 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.759921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.760000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.782859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.783050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.783391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.783587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.783833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.783961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.784033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.784250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.784389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.784658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.785042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.785411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.785811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.786201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.795602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.799147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.808305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.808538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.808738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.808979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.809203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.809380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.809609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.810274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.810846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.811088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.811323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.811542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.811747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.811957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:29:03.826922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.827142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:29:03.827506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.827636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.827708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:29:03.827814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:29:03.827974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.828101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.829588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:29:03.829687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.829768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.829857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.830871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.830871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.830871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.838203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.838479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.838749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.839031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.840585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.840865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.841272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.841566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.846440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.848858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.849340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.849626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.849911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.850184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.850462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.850730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.850967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.851280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.851664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.851789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.851950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.852096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.852182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.852295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.852589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.852879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.853145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.853420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.853698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.854086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.854364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.854643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.875114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.876586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.876870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.878944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.880485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.880776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.881517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.881805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.882095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.885436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.885723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.886010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.888606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.888893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.892482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.892778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.892924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.893222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.893503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.893789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.894062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.894335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.894603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.894872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.896854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.897166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.897459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.897750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.898031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.898292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.898561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.898828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.911866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.912144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.915976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.916122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.916414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.919148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.919494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.920510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.923654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.923783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.924026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.926759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.928234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.929984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.930256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.930565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.930846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.931129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.931376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.931504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.934634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.935031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.935736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.936030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.936314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.936596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.947519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.947824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.948089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.948365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.952513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.952812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.953088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.953357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.955334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.956842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.957125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.959743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.960025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.960257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.960386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.960603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.960880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.961778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.962060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.964718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.965014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.965313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.965608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.965893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.973432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.973715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.973992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.974266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.974546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.974819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.975108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.977266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.977558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.977842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.978124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.978404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.978691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.978976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.990638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.993667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.994316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.996568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.997424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:03.998235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:03.999728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.000014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.000287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.000379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.001733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.003221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.003512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.003810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.006666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.008136 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.010325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.010966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.011274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.011567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.011764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.011895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.014792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.015085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.015412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.015716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.019686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.019975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.020252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.023642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.023934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.024221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.027024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.027327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.027616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.027888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.028165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.028414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.028864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.029126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.029423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.029682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.029963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.030222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.030492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.030756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.031093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:29:04.031406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.031697 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.031985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.032263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.032540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.032819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.033100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.033388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.033661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.033948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.034225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.034503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.034781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:29:04.065055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639bccac9c0 pid: 81769 (default app) Aug 24 06:29:04.068525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a13c-7780-4a12-8844-9de4286d4f5f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:04.068769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639bccac9c0 pid: 81768 (default app) Aug 24 06:29:04.068921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a13c-7780-4a12-8844-9de4286d4f5f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:04.069045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a13c-7780-4a12-8844-9de4286d4f5f None None] PATH_INFO: `/v3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:04.073289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Aug 24 06:29:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:10.720911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536cc1a-a7f1-4a19-a7c7-cdfda9bbed88 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:10.721459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536cc1a-a7f1-4a19-a7c7-cdfda9bbed88 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:10.721459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536cc1a-a7f1-4a19-a7c7-cdfda9bbed88 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:10.725860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:10.733198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:10.733917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:10.734031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:10.808113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: INFO dbcounter [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] Registered counter for database keystone Aug 24 06:29:10.829885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG oslo_db.sqlalchemy.engines [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 06:29:10.841390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writer thread running {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 06:29:11.022489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG passlib.handlers.bcrypt [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81769) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 06:29:11.022925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG passlib.handlers.bcrypt [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81769) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 06:29:11.079808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:11.081436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:11.231814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 06:29:11.733908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536cc1a-a7f1-4a19-a7c7-cdfda9bbed88 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:11.813809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: INFO dbcounter [None req-4536cc1a-a7f1-4a19-a7c7-cdfda9bbed88 None None] Registered counter for database keystone Aug 24 06:29:11.839503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG oslo_db.sqlalchemy.engines [None req-4536cc1a-a7f1-4a19-a7c7-cdfda9bbed88 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81768) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 06:29:11.851754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writer thread running {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 06:29:11.975041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536cc1a-a7f1-4a19-a7c7-cdfda9bbed88 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:11.980517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846712d-1993-42b4-b3cc-5de4fcd9e963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:11.981143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846712d-1993-42b4-b3cc-5de4fcd9e963 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:11.981422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846712d-1993-42b4-b3cc-5de4fcd9e963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:11.981694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846712d-1993-42b4-b3cc-5de4fcd9e963 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:11.992827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846712d-1993-42b4-b3cc-5de4fcd9e963 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:12.053600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b846712d-1993-42b4-b3cc-5de4fcd9e963 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 06:29:12.053945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846712d-1993-42b4-b3cc-5de4fcd9e963 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:12.065226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Aug 24 06:29:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:29:12.075008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:12.096626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad8c33-4c54-4b23-9094-aec42ab9a637 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:12.102289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f6e04-8ee0-4328-a65c-c68b10d94d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:12.103461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f6e04-8ee0-4328-a65c-c68b10d94d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:12.103781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f6e04-8ee0-4328-a65c-c68b10d94d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:12.104068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f6e04-8ee0-4328-a65c-c68b10d94d4f admin admin] PATH_INFO: `/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:12.105804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f6e04-8ee0-4328-a65c-c68b10d94d4f admin admin] RBAC: Authorizing `identity:get_project(project_id=4d0ff1eb049e4a50aede95b2f05e94b7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:12.161172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING keystone.common.rbac_enforcer.enforcer [None req-450f6e04-8ee0-4328-a65c-c68b10d94d4f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 06:29:12.161519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f6e04-8ee0-4328-a65c-c68b10d94d4f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:12.166278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Aug 24 06:29:12 2022] GET /identity/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7 => generated 344 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:14.099812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a43c-b1e3-4495-9dc7-9044da24fa0e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:14.100814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a43c-b1e3-4495-9dc7-9044da24fa0e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:14.100814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a43c-b1e3-4495-9dc7-9044da24fa0e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:14.107728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:14.113597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8b16e-53e0-4775-95c9-3f0a54a69f00 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:14.114072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8b16e-53e0-4775-95c9-3f0a54a69f00 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:14.114185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8b16e-53e0-4775-95c9-3f0a54a69f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:14.136000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a6b8b16e-53e0-4775-95c9-3f0a54a69f00 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:14.137552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a6b8b16e-53e0-4775-95c9-3f0a54a69f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:14.146836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 06:29:14.417336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a43c-b1e3-4495-9dc7-9044da24fa0e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:14.438765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a43c-b1e3-4495-9dc7-9044da24fa0e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:14.443913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0c770-dd55-4999-bcf3-9b489e4dc632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:14.444547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c770-dd55-4999-bcf3-9b489e4dc632 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:14.444828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c770-dd55-4999-bcf3-9b489e4dc632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:14.445110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c770-dd55-4999-bcf3-9b489e4dc632 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:14.453710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0c770-dd55-4999-bcf3-9b489e4dc632 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:14.455563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0c770-dd55-4999-bcf3-9b489e4dc632 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:14.464394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Aug 24 06:29:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:29:14.470195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8b16e-53e0-4775-95c9-3f0a54a69f00 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:14.480555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8b16e-53e0-4775-95c9-3f0a54a69f00 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:14.486177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ece2-e710-4078-b763-a97d388a9dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:14.486819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ece2-e710-4078-b763-a97d388a9dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:14.487105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ece2-e710-4078-b763-a97d388a9dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:14.487419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ece2-e710-4078-b763-a97d388a9dbe admin admin] PATH_INFO: `/v3/users/66932a6f9a244fcaa7375a468f28c399` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:14.488886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ece2-e710-4078-b763-a97d388a9dbe admin admin] RBAC: Authorizing `identity:get_user(user_id=66932a6f9a244fcaa7375a468f28c399)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:14.490732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ece2-e710-4078-b763-a97d388a9dbe admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:14.492361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Aug 24 06:29:14 2022] GET /identity/v3/users/66932a6f9a244fcaa7375a468f28c399 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:16.472848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d8de1-cbe5-4ee7-bc25-86019fe0fff7 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:16.473361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d8de1-cbe5-4ee7-bc25-86019fe0fff7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:16.473361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d8de1-cbe5-4ee7-bc25-86019fe0fff7 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:16.474713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:16.480186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b556c8-0b01-4c96-921c-4e10e12eba3f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:16.480611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b556c8-0b01-4c96-921c-4e10e12eba3f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:16.480712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b556c8-0b01-4c96-921c-4e10e12eba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:16.500166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-02b556c8-0b01-4c96-921c-4e10e12eba3f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:16.501345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-02b556c8-0b01-4c96-921c-4e10e12eba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:16.512462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa96b-fcc7-4d1b-acd5-8366d569a490 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:16.512747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa96b-fcc7-4d1b-acd5-8366d569a490 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:16.513034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa96b-fcc7-4d1b-acd5-8366d569a490 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:16.515065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:16.520339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:16.520628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:16.520906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:16.566391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG passlib.handlers.bcrypt [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81768) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 06:29:16.566814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG passlib.handlers.bcrypt [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81768) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 06:29:16.585479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:16.614994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:16.616679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:16.717692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:16.860925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b556c8-0b01-4c96-921c-4e10e12eba3f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:16.886353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:16.970999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b556c8-0b01-4c96-921c-4e10e12eba3f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:16.979146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:17.005984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.006637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.006919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.007279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.019065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:17.027479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:17.028543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:17.032201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:17.034627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:17.087607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb843cdb-b5a1-4ad0-9a44-8edd6d3f9326 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:17.090901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:17.118402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:17.126753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.127675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.127967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.128251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.137255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156025b4-0c4e-47cc-bf82-bebfb34c7ad8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:17.175125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.175781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.176066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.176344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.177037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:17.177345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:17.178766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:17.192138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:17.197794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:17.198564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:17.199378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:17.304083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:29:17.306939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc205c9-7d17-416e-857b-8432c150a4f1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:17.309850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:17.313018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:17.343032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.343725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.344016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.344296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] PATH_INFO: `/v3/users/66932a6f9a244fcaa7375a468f28c399` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.345470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] RBAC: Authorizing `identity:get_user(user_id=66932a6f9a244fcaa7375a468f28c399)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:17.346951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:17.348576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:17 2022] GET /identity/v3/users/66932a6f9a244fcaa7375a468f28c399 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:17.354394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:17.369802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:17.417833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172d51e-c488-448a-9d15-a95bb7652b2d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:17.454232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.455100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.455412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.455687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.455782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7c26b-87d0-47fb-9f75-19f27d953e87 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:17.476132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:17.477813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:17.493898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.494548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.494835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.495115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.496288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:17.497752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:17.499432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:29:17.500633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:17.501692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:29:17.502533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:29:17.502533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:29:17.502533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:17.502533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:17.505512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:17.507143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:17.563502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:17.592981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:17.609461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03907981-c413-416d-97ac-7e11cf47bca8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:17.612705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020bedc-fc47-4f1c-b2b7-727c10a6d538 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:17.642832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6380ac9-57b6-4c44-acab-968e3a2ad7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.643417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6380ac9-57b6-4c44-acab-968e3a2ad7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.643712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6380ac9-57b6-4c44-acab-968e3a2ad7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.643985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6380ac9-57b6-4c44-acab-968e3a2ad7d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.644680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6380ac9-57b6-4c44-acab-968e3a2ad7d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:17.644952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6380ac9-57b6-4c44-acab-968e3a2ad7d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:17.646524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39c313-6996-45eb-a7ce-b8c2e3a34339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:17.646639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6380ac9-57b6-4c44-acab-968e3a2ad7d1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:17.647143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39c313-6996-45eb-a7ce-b8c2e3a34339 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:17.647471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39c313-6996-45eb-a7ce-b8c2e3a34339 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:17.647752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39c313-6996-45eb-a7ce-b8c2e3a34339 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:17.648487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39c313-6996-45eb-a7ce-b8c2e3a34339 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.domain.id=default, user.id=66932a6f9a244fcaa7375a468f28c399) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:17.648798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39c313-6996-45eb-a7ce-b8c2e3a34339 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:17.650229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39c313-6996-45eb-a7ce-b8c2e3a34339 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:17.657525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Aug 24 06:29:17 2022] GET /identity/v3/role_assignments?user.id=66932a6f9a244fcaa7375a468f28c399&scope.domain.id=default&role.id=c64ac97b1795434390dc18ab162aec39 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:17.666321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:19.713253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3a5d5-8481-4742-b3ec-61225ef1e9c1 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:19.713779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3a5d5-8481-4742-b3ec-61225ef1e9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:19.713779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3a5d5-8481-4742-b3ec-61225ef1e9c1 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:19.715167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:19.721655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f6620-68c4-41b7-86df-0f646019fcaf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:19.722521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f6620-68c4-41b7-86df-0f646019fcaf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:19.722994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f6620-68c4-41b7-86df-0f646019fcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:19.748240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bd8f6620-68c4-41b7-86df-0f646019fcaf None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:19.750549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bd8f6620-68c4-41b7-86df-0f646019fcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:19.847304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:20.043707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66025ebf-88f9-4d93-b298-75f8e5d1ab7d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.044353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66025ebf-88f9-4d93-b298-75f8e5d1ab7d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.044906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66025ebf-88f9-4d93-b298-75f8e5d1ab7d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.047056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:20.052317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c764ac0-e427-492b-8725-307ee688de15 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.052840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c764ac0-e427-492b-8725-307ee688de15 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.053354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c764ac0-e427-492b-8725-307ee688de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.080193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4c764ac0-e427-492b-8725-307ee688de15 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:20.081833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4c764ac0-e427-492b-8725-307ee688de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:20.141899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:20.168766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 12/25] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:20.277351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e27b9-19a4-41df-aea1-671636286547 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.277995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e27b9-19a4-41df-aea1-671636286547 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.278480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e27b9-19a4-41df-aea1-671636286547 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.280802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 14/26] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:20.286744 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2679c-f27f-4919-9132-e412eda3c565 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.287324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2679c-f27f-4919-9132-e412eda3c565 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.287835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2679c-f27f-4919-9132-e412eda3c565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.313971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bdb2679c-f27f-4919-9132-e412eda3c565 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:20.315636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bdb2679c-f27f-4919-9132-e412eda3c565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:20.319128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e27b9-19a4-41df-aea1-671636286547 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:20.497933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 13/27] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:20.552745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2679c-f27f-4919-9132-e412eda3c565 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:20.558875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e27b9-19a4-41df-aea1-671636286547 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:20.612148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53321de5-5dc5-4f9a-9f55-8e10acd3e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:20.612712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53321de5-5dc5-4f9a-9f55-8e10acd3e490 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.612997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53321de5-5dc5-4f9a-9f55-8e10acd3e490 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.613274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53321de5-5dc5-4f9a-9f55-8e10acd3e490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.639597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53321de5-5dc5-4f9a-9f55-8e10acd3e490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-53321de5-5dc5-4f9a-9f55-8e10acd3e490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:20.650771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:20.651678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:20.652447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:20.653536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 15/28] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:20.655059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08cef4-9f80-4df3-b203-96cf228a0105 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.655165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08cef4-9f80-4df3-b203-96cf228a0105 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.655807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08cef4-9f80-4df3-b203-96cf228a0105 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.656737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 16/29] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:20.661626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08cef4-9f80-4df3-b203-96cf228a0105 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:20.667094 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:20.780694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08cef4-9f80-4df3-b203-96cf228a0105 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:20.809407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2679c-f27f-4919-9132-e412eda3c565 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:20.817689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb2585-0f70-4bb3-961c-a12277b99c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:20.818245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb2585-0f70-4bb3-961c-a12277b99c6f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.818564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb2585-0f70-4bb3-961c-a12277b99c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.818848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb2585-0f70-4bb3-961c-a12277b99c6f None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.820457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb2585-0f70-4bb3-961c-a12277b99c6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:20.820784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb2585-0f70-4bb3-961c-a12277b99c6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:20.822192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb2585-0f70-4bb3-961c-a12277b99c6f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:20.826552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:20.832984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 17/30] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:20.834854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1fc4d-7199-43ae-abdd-1530a11c9802 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.835404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1fc4d-7199-43ae-abdd-1530a11c9802 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.836209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1fc4d-7199-43ae-abdd-1530a11c9802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.858313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-92a1fc4d-7199-43ae-abdd-1530a11c9802 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:20.859898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-92a1fc4d-7199-43ae-abdd-1530a11c9802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:20.890369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:20.890916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:20.891223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:20.891533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:20.892262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:20.893666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:20.939411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:20.948027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 18/31] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:20.949177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1fc4d-7199-43ae-abdd-1530a11c9802 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:21.057869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1fc4d-7199-43ae-abdd-1530a11c9802 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:21.095539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:21.096563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.097039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.097509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] PATH_INFO: `/v3/users/66932a6f9a244fcaa7375a468f28c399` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.098944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] RBAC: Authorizing `identity:get_user(user_id=66932a6f9a244fcaa7375a468f28c399)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:21.100656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:21.102482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 19/32] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:20 2022] GET /identity/v3/users/66932a6f9a244fcaa7375a468f28c399 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:21.103534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:21.104898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 14/33] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 06:29:20 2022] POST /identity/v3/roles => generated 219 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:21.111831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:21.218634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:21.339657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d000168-e36f-47cd-9424-00a8dac4114a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:21.384718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:21.385366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.385828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.386258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.397318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:21.399027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:21.404795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41942-a7f4-4de4-8d4e-fedcfa6743dc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:29:21.417397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:21.418494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:29:21.419519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:29:21.419519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:29:21.419519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:21.421003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 20/34] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:21.422036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:21.445043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6acc-8399-4bbd-94f5-d7e9583623a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:21.446079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6acc-8399-4bbd-94f5-d7e9583623a2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.446606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6acc-8399-4bbd-94f5-d7e9583623a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.447060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6acc-8399-4bbd-94f5-d7e9583623a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.448788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6acc-8399-4bbd-94f5-d7e9583623a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:21.450705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6acc-8399-4bbd-94f5-d7e9583623a2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:21.452806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 15/35] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:21.454354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c141-ad75-415b-9407-57eb87e3931f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.454833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c141-ad75-415b-9407-57eb87e3931f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.455480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819c141-ad75-415b-9407-57eb87e3931f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.456954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 16/36] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:21.457746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819c141-ad75-415b-9407-57eb87e3931f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:21.502126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:21.540957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f827e2-4ae7-4dc7-8e91-9933324a6c8b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:21.560289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:21.577676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819c141-ad75-415b-9407-57eb87e3931f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:21.583889 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf91d07-061e-459b-b808-23557afe3f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:21.584510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91d07-061e-459b-b808-23557afe3f77 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.584790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91d07-061e-459b-b808-23557afe3f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.585050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf91d07-061e-459b-b808-23557afe3f77 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.585734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf91d07-061e-459b-b808-23557afe3f77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:21.587244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf91d07-061e-459b-b808-23557afe3f77 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:21.616157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:21.616779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.617064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.617390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.618044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:21.618374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:21.619830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:21.631875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 17/37] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:21.632518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:21.709409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 21/38] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 06:29:21 2022] POST /identity/v3/roles => generated 225 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:21.710848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd1ef-76a6-4448-beef-7787e8ca04b8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.711139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd1ef-76a6-4448-beef-7787e8ca04b8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.711458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd1ef-76a6-4448-beef-7787e8ca04b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.728690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:21.733948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d4ebd1ef-76a6-4448-beef-7787e8ca04b8 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:21.736275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d4ebd1ef-76a6-4448-beef-7787e8ca04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:21.737823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86995bc0-a377-4574-b342-2b416d1c7bb5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:21.777701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c232fcf-b71e-4ce5-9db9-203f76c3d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:21.778310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c232fcf-b71e-4ce5-9db9-203f76c3d436 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.778604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c232fcf-b71e-4ce5-9db9-203f76c3d436 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.778896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c232fcf-b71e-4ce5-9db9-203f76c3d436 None admin] PATH_INFO: `/v3/domains/default/users/66932a6f9a244fcaa7375a468f28c399/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.781601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c232fcf-b71e-4ce5-9db9-203f76c3d436 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=66932a6f9a244fcaa7375a468f28c399, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:21.783339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c232fcf-b71e-4ce5-9db9-203f76c3d436 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:21.815223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 18/39] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Aug 24 06:29:21 2022] PUT /identity/v3/domains/default/users/66932a6f9a244fcaa7375a468f28c399/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:21.815223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe175f74-f2fe-45fc-af62-30b6acfec8ba None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.815223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe175f74-f2fe-45fc-af62-30b6acfec8ba None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.815223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe175f74-f2fe-45fc-af62-30b6acfec8ba None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.815223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 19/40] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:21.815223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe175f74-f2fe-45fc-af62-30b6acfec8ba None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:21.825552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 22/41] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:21.836829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d61392-bf3d-4b22-aab8-3f0fd21d144b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:21.837228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d61392-bf3d-4b22-aab8-3f0fd21d144b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:21.837578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d61392-bf3d-4b22-aab8-3f0fd21d144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:21.858278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-43d61392-bf3d-4b22-aab8-3f0fd21d144b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:21.859679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-43d61392-bf3d-4b22-aab8-3f0fd21d144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:21.945220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 23/42] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:21.962893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe175f74-f2fe-45fc-af62-30b6acfec8ba None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:22.005136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:22.006111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:22.006576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:22.007052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:22.007920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:22.009655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:22.044378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 20/43] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:29:21 2022] POST /identity/v3/projects => generated 301 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:22.128808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d61392-bf3d-4b22-aab8-3f0fd21d144b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:22.166441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:22.240335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:22.247720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d61392-bf3d-4b22-aab8-3f0fd21d144b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:22.253753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:22.294292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:22.296454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:22.296683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:22.296966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:22.297662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:22.299230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:22.316581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 24/44] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 06:29:22 2022] POST /identity/v3/roles => generated 223 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:22.383019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a7ebd-97d3-4cec-8a9d-05c62b611bad None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:22.421370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f6b0d-71a7-438a-93a9-beb39b42cef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:22.422039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6b0d-71a7-438a-93a9-beb39b42cef0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:22.422331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6b0d-71a7-438a-93a9-beb39b42cef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:22.422613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6b0d-71a7-438a-93a9-beb39b42cef0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:22.423812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f6b0d-71a7-438a-93a9-beb39b42cef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:22.425314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f6b0d-71a7-438a-93a9-beb39b42cef0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:22.426982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 21/45] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:22.433106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:22.650684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:22.657216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab14ba-4805-4da5-9dbb-ea682156c5b4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:22.692317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360fb3d-1f14-4b25-b4b2-ac9575e480fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:22.692930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360fb3d-1f14-4b25-b4b2-ac9575e480fb None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:22.693221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360fb3d-1f14-4b25-b4b2-ac9575e480fb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:22.693499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360fb3d-1f14-4b25-b4b2-ac9575e480fb None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:22.694181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360fb3d-1f14-4b25-b4b2-ac9575e480fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:22.695687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360fb3d-1f14-4b25-b4b2-ac9575e480fb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:22.725237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 25/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:29:22 2022] POST /identity/v3/projects => generated 312 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:29:23.894508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe5f7f-db08-4b4d-bc2d-2144f9f5e432 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:23.895093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe5f7f-db08-4b4d-bc2d-2144f9f5e432 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:23.895093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe5f7f-db08-4b4d-bc2d-2144f9f5e432 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:23.896484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 22/47] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:23.902212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4df36-7127-469a-9da9-265d118dbf95 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:23.902618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4df36-7127-469a-9da9-265d118dbf95 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:23.902788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4df36-7127-469a-9da9-265d118dbf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:23.922766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5bc4df36-7127-469a-9da9-265d118dbf95 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:23.924061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5bc4df36-7127-469a-9da9-265d118dbf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:23.993936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 26/48] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:24.259957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe5f7f-db08-4b4d-bc2d-2144f9f5e432 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:24.342755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:24.370566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe5f7f-db08-4b4d-bc2d-2144f9f5e432 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:24.405311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:24.405947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:24.406234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:24.406509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:24.418028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:24.424054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:24.424895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:24.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 23/49] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:24.431307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4df36-7127-469a-9da9-265d118dbf95 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:24.518833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:24.542181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4df36-7127-469a-9da9-265d118dbf95 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:24.578286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609940d-f4bf-480e-88e8-5eb6624b1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:24.578873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609940d-f4bf-480e-88e8-5eb6624b1abf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:24.579175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609940d-f4bf-480e-88e8-5eb6624b1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:24.579528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609940d-f4bf-480e-88e8-5eb6624b1abf None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:24.580262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609940d-f4bf-480e-88e8-5eb6624b1abf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:24.580545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609940d-f4bf-480e-88e8-5eb6624b1abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:24.582139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609940d-f4bf-480e-88e8-5eb6624b1abf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:24.590597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 27/50] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:24.597212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:24.620405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f607a5-862a-4224-92e5-e4a77840b96a None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:24.620693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f607a5-862a-4224-92e5-e4a77840b96a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:24.620973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f607a5-862a-4224-92e5-e4a77840b96a None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:24.622481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 28/51] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:24.628192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9ad29-e10e-4246-afc4-8b6e6ebeae67 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:24.628486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9ad29-e10e-4246-afc4-8b6e6ebeae67 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:24.628770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9ad29-e10e-4246-afc4-8b6e6ebeae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:24.648116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-61b9ad29-e10e-4246-afc4-8b6e6ebeae67 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:24.649316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-61b9ad29-e10e-4246-afc4-8b6e6ebeae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:24.705477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6477c52-45d4-4ce2-92ff-ef819ee2d93e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:24.720951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 29/52] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:24.739934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:24.740582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:24.740866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:24.741151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] PATH_INFO: `/v3/users/66932a6f9a244fcaa7375a468f28c399` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:24.742323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] RBAC: Authorizing `identity:get_user(user_id=66932a6f9a244fcaa7375a468f28c399)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:24.743828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:24.745428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 24/53] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:24 2022] GET /identity/v3/users/66932a6f9a244fcaa7375a468f28c399 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:24.751266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9ad29-e10e-4246-afc4-8b6e6ebeae67 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:24.821640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:24.854983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9ad29-e10e-4246-afc4-8b6e6ebeae67 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:24.890833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:24.891503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:24.891795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:24.892074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:24.893249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:24.894726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:24.896377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 30/54] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:24.902369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:24.905626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:24.986479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:25.010138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4234c24-1915-421a-a4fa-2a029b55b953 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:25.045415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:25.046075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:25.046366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:25.046641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:25.047421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.domain.id=default, user.id=66932a6f9a244fcaa7375a468f28c399) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:25.047737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:25.049478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:25.058263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 25/55] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Aug 24 06:29:24 2022] GET /identity/v3/role_assignments?user.id=66932a6f9a244fcaa7375a468f28c399&scope.domain.id=default&role.id=c64ac97b1795434390dc18ab162aec39 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:25.093872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33328cff-6bce-4d18-9cf0-3f76fd67263d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:25.132539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661d1b4-6cbb-41eb-8032-63e80e7132d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:25.133186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661d1b4-6cbb-41eb-8032-63e80e7132d2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:25.133469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661d1b4-6cbb-41eb-8032-63e80e7132d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:25.133745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661d1b4-6cbb-41eb-8032-63e80e7132d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:25.134926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661d1b4-6cbb-41eb-8032-63e80e7132d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:25.136473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661d1b4-6cbb-41eb-8032-63e80e7132d2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:25.138351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 31/56] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:25.144298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:25.222097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:25.264556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569a3c6-6d97-41d8-bf02-a9e9a3a6ada6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:25.297857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8870f-2b2c-483c-9d38-be79e430323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:25.298495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8870f-2b2c-483c-9d38-be79e430323c None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:25.298782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8870f-2b2c-483c-9d38-be79e430323c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:25.299056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8870f-2b2c-483c-9d38-be79e430323c None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:25.299757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8870f-2b2c-483c-9d38-be79e430323c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:25.301197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8870f-2b2c-483c-9d38-be79e430323c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:25.329199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 26/57] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Aug 24 06:29:25 2022] POST /identity/v3/projects => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:27.242824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1cb4-e876-4592-9b78-713da8453873 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:27.243323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1cb4-e876-4592-9b78-713da8453873 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:27.243438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f1cb4-e876-4592-9b78-713da8453873 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:27.244781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 32/58] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:27.251253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a04f5-f901-490e-92e4-bd44888ec1c5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:27.251951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a04f5-f901-490e-92e4-bd44888ec1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:27.252125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a04f5-f901-490e-92e4-bd44888ec1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:27.283096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-cd0a04f5-f901-490e-92e4-bd44888ec1c5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:27.284653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cd0a04f5-f901-490e-92e4-bd44888ec1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:27.355407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 27/59] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:27.622402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1cb4-e876-4592-9b78-713da8453873 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:27.632770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:27.735743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f1cb4-e876-4592-9b78-713da8453873 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:27.776148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e61ed7-6213-4fbe-872a-e65ba937bb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:27.777053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e61ed7-6213-4fbe-872a-e65ba937bb0a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:27.777416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e61ed7-6213-4fbe-872a-e65ba937bb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:27.777759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e61ed7-6213-4fbe-872a-e65ba937bb0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:27.779956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e61ed7-6213-4fbe-872a-e65ba937bb0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:27.781727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e61ed7-6213-4fbe-872a-e65ba937bb0a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:27.783348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 33/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:27.789159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a04f5-f901-490e-92e4-bd44888ec1c5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:27.851614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:27.903397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a04f5-f901-490e-92e4-bd44888ec1c5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:27.942445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46974a-0ca4-473c-a8fa-5d93eb288dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:27.943061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46974a-0ca4-473c-a8fa-5d93eb288dec None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:27.943378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46974a-0ca4-473c-a8fa-5d93eb288dec None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:27.943658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46974a-0ca4-473c-a8fa-5d93eb288dec None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:27.944316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46974a-0ca4-473c-a8fa-5d93eb288dec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:27.945799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46974a-0ca4-473c-a8fa-5d93eb288dec None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:27.975235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 28/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:29:27 2022] POST /identity/v3/users => generated 275 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:30.835970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb968507-1fdc-4fa1-bac9-f80c5a15cff6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:30.837018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb968507-1fdc-4fa1-bac9-f80c5a15cff6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:30.837495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb968507-1fdc-4fa1-bac9-f80c5a15cff6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:30.839054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 34/62] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:30.845034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee5115-ddad-49eb-ad9d-14c21438c8f6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:30.845501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee5115-ddad-49eb-ad9d-14c21438c8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:30.845623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee5115-ddad-49eb-ad9d-14c21438c8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:30.868170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ddee5115-ddad-49eb-ad9d-14c21438c8f6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:30.869606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ddee5115-ddad-49eb-ad9d-14c21438c8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:30.961537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 29/63] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:31.342372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb1d21-b3c7-42c9-8556-7eaab7d337c6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.343327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb1d21-b3c7-42c9-8556-7eaab7d337c6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.343813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb1d21-b3c7-42c9-8556-7eaab7d337c6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.345369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 35/64] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:31.350870 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d907c8f-686e-477e-8a83-9e81586ed9cf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.351455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d907c8f-686e-477e-8a83-9e81586ed9cf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.351934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d907c8f-686e-477e-8a83-9e81586ed9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.377158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6d907c8f-686e-477e-8a83-9e81586ed9cf None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:31.378704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6d907c8f-686e-477e-8a83-9e81586ed9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:31.392886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb1d21-b3c7-42c9-8556-7eaab7d337c6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:31.407425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:31.466274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 30/65] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:31.468203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea8cf6-3866-43d9-b95e-a2c369a459c4 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.468545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea8cf6-3866-43d9-b95e-a2c369a459c4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.468827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea8cf6-3866-43d9-b95e-a2c369a459c4 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.470097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 31/66] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:31.476636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd205b-5a7f-4386-8d84-f2a4fbee916c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.477150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd205b-5a7f-4386-8d84-f2a4fbee916c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.477623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd205b-5a7f-4386-8d84-f2a4fbee916c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.496352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-54bd205b-5a7f-4386-8d84-f2a4fbee916c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:31.497908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-54bd205b-5a7f-4386-8d84-f2a4fbee916c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:31.539939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:31.570177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 32/67] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:31.590161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb1d21-b3c7-42c9-8556-7eaab7d337c6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:31.626723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f7a01-613c-4b6b-850e-20b1227d3bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:31.627737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f7a01-613c-4b6b-850e-20b1227d3bfe None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.628307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f7a01-613c-4b6b-850e-20b1227d3bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.628779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f7a01-613c-4b6b-850e-20b1227d3bfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.640874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f7a01-613c-4b6b-850e-20b1227d3bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5a6f7a01-613c-4b6b-850e-20b1227d3bfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:31.646303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:31.647277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:31.648091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:31.650391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 36/68] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:31.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd205b-5a7f-4386-8d84-f2a4fbee916c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:31.672132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e09acf-e1a0-44fe-9fe1-d5440ff50e5c None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.672725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e09acf-e1a0-44fe-9fe1-d5440ff50e5c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.673201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e09acf-e1a0-44fe-9fe1-d5440ff50e5c None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.674840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 37/69] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:31.680418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af24fb-2b42-4991-b5b0-7f16ae5fb365 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.680948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af24fb-2b42-4991-b5b0-7f16ae5fb365 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.681425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af24fb-2b42-4991-b5b0-7f16ae5fb365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.712781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f4af24fb-2b42-4991-b5b0-7f16ae5fb365 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:31.715671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f4af24fb-2b42-4991-b5b0-7f16ae5fb365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:31.785274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:31.793599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 38/70] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:31.794448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd205b-5a7f-4386-8d84-f2a4fbee916c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:31.795305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c281-8d54-4f92-b22c-dccfe17366b7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.795718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c281-8d54-4f92-b22c-dccfe17366b7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.796124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c281-8d54-4f92-b22c-dccfe17366b7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.799229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 39/71] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:31.799229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2c281-8d54-4f92-b22c-dccfe17366b7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:31.841376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbe22d-fa46-4385-9aae-b1b8b4056afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:31.842113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbe22d-fa46-4385-9aae-b1b8b4056afd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.842575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbe22d-fa46-4385-9aae-b1b8b4056afd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.842890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbe22d-fa46-4385-9aae-b1b8b4056afd None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.843556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbe22d-fa46-4385-9aae-b1b8b4056afd None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:31.843914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbe22d-fa46-4385-9aae-b1b8b4056afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:31.845454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbbe22d-fa46-4385-9aae-b1b8b4056afd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:31.859143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 33/72] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:31.860692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaae29-69bd-40b4-922c-8c983aebe774 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.861147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaae29-69bd-40b4-922c-8c983aebe774 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.861596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaae29-69bd-40b4-922c-8c983aebe774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.872012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:31.916049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2c281-8d54-4f92-b22c-dccfe17366b7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:31.955632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:31.956298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:31.956583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:31.956860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:31.969646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:31.974559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:31.975496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 40/73] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:31.980069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:31.997652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-70eaae29-69bd-40b4-922c-8c983aebe774 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:31.999240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-70eaae29-69bd-40b4-922c-8c983aebe774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:32.069957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 34/74] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:32.070694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaae29-69bd-40b4-922c-8c983aebe774 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.092373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0cd6f-a7a9-4c6a-865c-0d3c431632b7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.122652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.134760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.135929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.136494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.137109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.151556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.153026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.159279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 41/75] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:31 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:32.159279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.171461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.188264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaae29-69bd-40b4-922c-8c983aebe774 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.228578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.229226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.229512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.229784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.241287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:32.246196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:32.247082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:32.247935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 35/76] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:32.248639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.273825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f53806-71a0-4fb0-8046-cc10c3f374ba None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.311115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.312212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.312720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.312824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.313482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:32.313806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.315224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.329248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 42/77] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:32.329248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.365212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba99db-2b2d-48bf-91e4-206be0731ba0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.375108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.399618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.400258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.400546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.400820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.410807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:32.415614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:32.416446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:32.417230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 36/78] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:32.417937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.427131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.431718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e1db4-4303-47a3-a784-0ad64ccfe037 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.465646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.466372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.466770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.467146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.520242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0699e82-12f4-4eb1-996b-9467c2c216a8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.534540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.536113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.555297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.555939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.556225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.556501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.557180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:32.557490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.558880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.569385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 37/79] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:32.570242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.593848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.641320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.648710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 43/80] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:32.652167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.677038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdd929-579a-4e75-96b5-71e1f2810bc5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.710940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.711626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.711918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.712254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] PATH_INFO: `/v3/users/66932a6f9a244fcaa7375a468f28c399` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.713417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] RBAC: Authorizing `identity:get_user(user_id=66932a6f9a244fcaa7375a468f28c399)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.714891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.716621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 38/81] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/users/66932a6f9a244fcaa7375a468f28c399 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:32.717397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.752628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f112b3b-2aa4-497b-9add-c088f22f9d93 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.785390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.786025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.786308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.786582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.787719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.789182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.790758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 44/82] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:32.791542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.806530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.816946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6237ef-a4ab-4b0f-b777-19c815726ae1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.851499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.852103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.852308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.852583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.853265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:32.853576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.854966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.857184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:32.864292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 39/83] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:32.865011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.898089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b78632-8f73-41bb-a10e-35e20f83e231 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.932637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:32.933266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:32.933523 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:32.933819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:32.934998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:32.936578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:32.938511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 45/84] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:32.939300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:32.965015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282809a-75f7-4483-9261-f75db3f4802b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:32.999342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.000010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.000296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.000577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.001728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.003184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.005092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 40/85] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:33.005748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:33.022101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.042575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248bd39-5b97-42f3-987b-e150b89e8f0c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.076124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.076572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.077219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.077538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.077890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.112559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448a51c-06a6-4587-8bce-b0862f2a278d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.145341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.145957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.146246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.146529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.146800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.147182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.148330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.148647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.171088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 41/86] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:29:33 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:33.172027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:33.248611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.272685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 46/87] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:33.273574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:33.299005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d6072-4675-48b7-981a-3e39fddf55b3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.328184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.346116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.347270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.347569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.347856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.349389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.351167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.352925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 42/88] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:33 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:33.352925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:33.391316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70033865-00d7-4e40-b910-82ce5eac3a78 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.433656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.434311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.434595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.435156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.477354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b182d37-262b-4683-8584-b5f0301a45cf None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.507036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.527953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.529263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.529556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.529729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.529853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.530647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=636cfdf20b5042e8a116abd580b78f06) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:33.531010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.531112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.532542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.539978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 43/89] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:33 2022] GET /identity/v3/role_assignments?user.id=636cfdf20b5042e8a116abd580b78f06&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:33.540620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:33.606829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.660661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1f6cd-3e3e-40cf-9ed5-ad391dfe872d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.701000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.701732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.702004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.702284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.703977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.705733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.707832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 44/90] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:33 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:33.708737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:33.792439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 47/91] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:33.793538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:33.901187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51820dac-045c-448c-9339-9aced8e0231c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.930377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.954068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27ebd4-13ef-4a0a-9a89-73c6e510e07d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:33.969448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:33.994213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.995026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.995337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.995615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:33.996361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=66932a6f9a244fcaa7375a468f28c399) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:33.996680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:33.997588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:33.998140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:33.998292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:33.998547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:33.999504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:34.005867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 48/92] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:33 2022] GET /identity/v3/role_assignments?user.id=66932a6f9a244fcaa7375a468f28c399&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:34.006525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:34.151923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:34.154212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:34.293282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a52238-871a-48e5-a675-1a6276a3cf6f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:34.362424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 45/93] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:34.372284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:34.426480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:34.484121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:34.484713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:34.485009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:34.485298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:34.486509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:34.488033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:34.489887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 49/94] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:34 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:34.497125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:34.537256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:34.625211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1339b-aa45-4449-94a1-0fe35938cc24 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:34.689029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:34.689698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:34.689987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:34.690263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] PATH_INFO: `/v3/projects/f221fd0545b14ea1adf232f3dfccadda` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:34.695922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] RBAC: Authorizing `identity:get_project(project_id=f221fd0545b14ea1adf232f3dfccadda)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:34.697400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:34.699126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 46/95] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:34 2022] GET /identity/v3/projects/f221fd0545b14ea1adf232f3dfccadda => generated 312 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:34.709723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:34.782149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001f45-ff1f-4222-863a-b7df3b1f0e96 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:34.834406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ab560-ad9b-4623-9ccd-a3a8881a571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:34.835036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ab560-ad9b-4623-9ccd-a3a8881a571f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:34.835360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ab560-ad9b-4623-9ccd-a3a8881a571f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:34.835643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ab560-ad9b-4623-9ccd-a3a8881a571f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:34.836399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ab560-ad9b-4623-9ccd-a3a8881a571f None admin] RBAC: Adding query filter params (role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=636cfdf20b5042e8a116abd580b78f06) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:34.836726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ab560-ad9b-4623-9ccd-a3a8881a571f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:34.838274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ab560-ad9b-4623-9ccd-a3a8881a571f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:34.839020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:34.845940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 50/96] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:34 2022] GET /identity/v3/role_assignments?user.id=636cfdf20b5042e8a116abd580b78f06&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 259 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:34.863882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafaa1c-f0b8-44d2-8a41-731260607bae None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:34.931388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e250022-7b0e-4deb-bd70-fccb50185303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:34.932031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250022-7b0e-4deb-bd70-fccb50185303 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:34.932281 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250022-7b0e-4deb-bd70-fccb50185303 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:34.932556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250022-7b0e-4deb-bd70-fccb50185303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:34.933306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e250022-7b0e-4deb-bd70-fccb50185303 None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=f221fd0545b14ea1adf232f3dfccadda, user.id=636cfdf20b5042e8a116abd580b78f06) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:34.933616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e250022-7b0e-4deb-bd70-fccb50185303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:34.935358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e250022-7b0e-4deb-bd70-fccb50185303 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:34.948025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 47/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:34 2022] GET /identity/v3/role_assignments?user.id=636cfdf20b5042e8a116abd580b78f06&scope.project.id=f221fd0545b14ea1adf232f3dfccadda&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57801-f7b2-4fea-8d10-1ce108314a9b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:35.794359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57801-f7b2-4fea-8d10-1ce108314a9b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:35.794848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57801-f7b2-4fea-8d10-1ce108314a9b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:35.796692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 51/98] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:35.802604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973d3a5-1c94-4c73-b164-0bd50b7944a4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:35.803226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973d3a5-1c94-4c73-b164-0bd50b7944a4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:35.803708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973d3a5-1c94-4c73-b164-0bd50b7944a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:35.832158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2973d3a5-1c94-4c73-b164-0bd50b7944a4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:35.833746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2973d3a5-1c94-4c73-b164-0bd50b7944a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:35.903634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 48/99] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:35.979597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b6cf1-b146-45bc-9baa-5447260d9fce None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:35.979886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b6cf1-b146-45bc-9baa-5447260d9fce None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:35.980407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b6cf1-b146-45bc-9baa-5447260d9fce None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:35.981973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 52/100] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:35.988182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8fb66-ff38-41f9-b516-dc79f5ea6a93 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:35.988656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8fb66-ff38-41f9-b516-dc79f5ea6a93 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:35.988752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8fb66-ff38-41f9-b516-dc79f5ea6a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:36.016232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-08a8fb66-ff38-41f9-b516-dc79f5ea6a93 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:36.017712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-08a8fb66-ff38-41f9-b516-dc79f5ea6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:36.097923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 49/101] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:36.369567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b6cf1-b146-45bc-9baa-5447260d9fce None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:36.463420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:36.497027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b6cf1-b146-45bc-9baa-5447260d9fce None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:36.540882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:36.541542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:36.541828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:36.542107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:36.561610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:36.567429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:36.568287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:36.569154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 53/102] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:36.576762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8fb66-ff38-41f9-b516-dc79f5ea6a93 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:36.580051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:36.598523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:36.706375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67611c80-1448-4b5a-b520-9a9fefc790b4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:36.768557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8fb66-ff38-41f9-b516-dc79f5ea6a93 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:36.788644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c1fb1-0ca9-4891-b29c-7d68fe4f974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:36.789771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1fb1-0ca9-4891-b29c-7d68fe4f974f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:36.791490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1fb1-0ca9-4891-b29c-7d68fe4f974f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:36.792411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1fb1-0ca9-4891-b29c-7d68fe4f974f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:36.794228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c1fb1-0ca9-4891-b29c-7d68fe4f974f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:36.797274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c1fb1-0ca9-4891-b29c-7d68fe4f974f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:36.801119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 54/103] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:36.803778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52ff2-61fc-4d8d-81b7-cefded56550b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:36.804447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52ff2-61fc-4d8d-81b7-cefded56550b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:36.806316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52ff2-61fc-4d8d-81b7-cefded56550b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:36.808495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 55/104] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:36.810012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52ff2-61fc-4d8d-81b7-cefded56550b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:36.817888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:36.831241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d88f3-e854-4724-b35a-740791de3a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:36.833114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d88f3-e854-4724-b35a-740791de3a6c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:36.834610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d88f3-e854-4724-b35a-740791de3a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:36.835083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d88f3-e854-4724-b35a-740791de3a6c None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:36.836021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d88f3-e854-4724-b35a-740791de3a6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:36.837670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d88f3-e854-4724-b35a-740791de3a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:36.839282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d88f3-e854-4724-b35a-740791de3a6c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:36.852946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 50/105] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:36.854596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bbbe9-fc4d-47ea-8ffe-d871f268b071 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:36.855077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bbbe9-fc4d-47ea-8ffe-d871f268b071 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:36.855563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bbbe9-fc4d-47ea-8ffe-d871f268b071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:36.879260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-490bbbe9-fc4d-47ea-8ffe-d871f268b071 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:36.880781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-490bbbe9-fc4d-47ea-8ffe-d871f268b071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:36.965020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 51/106] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:36.966191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bbbe9-fc4d-47ea-8ffe-d871f268b071 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:36.975702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:36.976886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52ff2-61fc-4d8d-81b7-cefded56550b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:37.058470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bdb3c-9584-40d6-b830-40d9270e014b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:37.062947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdb3c-9584-40d6-b830-40d9270e014b None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.063421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdb3c-9584-40d6-b830-40d9270e014b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.063849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdb3c-9584-40d6-b830-40d9270e014b None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.064749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bdb3c-9584-40d6-b830-40d9270e014b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:37.066397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bdb3c-9584-40d6-b830-40d9270e014b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:37.079270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bbbe9-fc4d-47ea-8ffe-d871f268b071 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:37.118993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fd1ec-da8c-4a49-8980-bd188eea0c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:37.120001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fd1ec-da8c-4a49-8980-bd188eea0c13 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.120501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fd1ec-da8c-4a49-8980-bd188eea0c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.120918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fd1ec-da8c-4a49-8980-bd188eea0c13 None admin] PATH_INFO: `/v3/users/89bf34b8d13d4dea8257337bbdd45ace` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.137140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 56/107] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:29:36 2022] POST /identity/v3/projects => generated 302 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:37.138217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fd1ec-da8c-4a49-8980-bd188eea0c13 None admin] RBAC: Authorizing `identity:get_user(user_id=89bf34b8d13d4dea8257337bbdd45ace)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:37.139745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3507a-7efd-4a4d-b436-b03c9fe752a3 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.140110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3507a-7efd-4a4d-b436-b03c9fe752a3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.140395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3507a-7efd-4a4d-b436-b03c9fe752a3 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.140730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fd1ec-da8c-4a49-8980-bd188eea0c13 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:37.141713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 57/108] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:37.142977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 52/109] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:36 2022] GET /identity/v3/users/89bf34b8d13d4dea8257337bbdd45ace => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:37.149024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3507a-7efd-4a4d-b436-b03c9fe752a3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:37.150705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb9bf4-2c29-4b3b-8d24-365a5ac6a426 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.151703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb9bf4-2c29-4b3b-8d24-365a5ac6a426 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.152587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb9bf4-2c29-4b3b-8d24-365a5ac6a426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.189135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2dbb9bf4-2c29-4b3b-8d24-365a5ac6a426 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:37.191691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2dbb9bf4-2c29-4b3b-8d24-365a5ac6a426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:37.293024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:37.315389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:37.324487 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 53/110] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:37.354563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3507a-7efd-4a4d-b436-b03c9fe752a3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:37.401072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c6eb6-3a00-42f0-bd81-0eee73570915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:37.402092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6eb6-3a00-42f0-bd81-0eee73570915 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.402631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6eb6-3a00-42f0-bd81-0eee73570915 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.403110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c6eb6-3a00-42f0-bd81-0eee73570915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.494200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c6eb6-3a00-42f0-bd81-0eee73570915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:37.496069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c6eb6-3a00-42f0-bd81-0eee73570915 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:37.516195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb9bf4-2c29-4b3b-8d24-365a5ac6a426 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:37.600799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:37.650832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 58/111] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:37.654637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed92816-2bc2-45b8-8897-1cf2cb20a62f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.654637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed92816-2bc2-45b8-8897-1cf2cb20a62f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.654637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed92816-2bc2-45b8-8897-1cf2cb20a62f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.654637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 59/112] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:37.660074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd397f-9678-40ea-8f5e-ad372e957e83 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.660363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd397f-9678-40ea-8f5e-ad372e957e83 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.660642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd397f-9678-40ea-8f5e-ad372e957e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.668514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb9bf4-2c29-4b3b-8d24-365a5ac6a426 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:37.680744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a3fd397f-9678-40ea-8f5e-ad372e957e83 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:37.681948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a3fd397f-9678-40ea-8f5e-ad372e957e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:37.710867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:37.712518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.713026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.713730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.727673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:37.735021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:37.735981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:37.736760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:37.739265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 54/113] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:37.740537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:37.761673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 60/114] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:37.762590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd397f-9678-40ea-8f5e-ad372e957e83 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:37.808830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:37.863950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f3431-bf49-4fb1-9c4d-d66044477f08 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:37.875553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:37.884685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd397f-9678-40ea-8f5e-ad372e957e83 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:37.903870 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:37.904909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.905361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.905949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.907316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:37.908973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:37.910916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 55/115] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:37 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:37.911923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:37.928924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a6de1-f234-4474-a583-087f6df63533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:37.929591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a6de1-f234-4474-a583-087f6df63533 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.930140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a6de1-f234-4474-a583-087f6df63533 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.930485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a6de1-f234-4474-a583-087f6df63533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.945374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a6de1-f234-4474-a583-087f6df63533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-297a6de1-f234-4474-a583-087f6df63533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:37.950425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:37.951328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:37.952378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:37.953047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 61/116] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:37.954080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442a35-e1e6-4840-b17f-eed325784544 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:37.954370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442a35-e1e6-4840-b17f-eed325784544 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:37.954689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4442a35-e1e6-4840-b17f-eed325784544 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:37.959981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 62/117] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:37.959981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4442a35-e1e6-4840-b17f-eed325784544 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.051993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d16ce-358a-4f4f-abba-3bfe2d4872ec None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.075913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4442a35-e1e6-4840-b17f-eed325784544 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.078074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:38.092537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.093209 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.093526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.093717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.094474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:38.094711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:38.096178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:38.105060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 56/118] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:38.105864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.117316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735688-6314-4cff-a98b-6f3c7bc54481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.118060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735688-6314-4cff-a98b-6f3c7bc54481 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.118444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735688-6314-4cff-a98b-6f3c7bc54481 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.118795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735688-6314-4cff-a98b-6f3c7bc54481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.119577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735688-6314-4cff-a98b-6f3c7bc54481 None admin] RBAC: Adding query filter params (role.id=1953445516ad4bdf957eed35d5c68504, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=89bf34b8d13d4dea8257337bbdd45ace) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:38.119889 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735688-6314-4cff-a98b-6f3c7bc54481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:38.121351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735688-6314-4cff-a98b-6f3c7bc54481 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:38.135217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 63/119] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:37 2022] GET /identity/v3/role_assignments?user.id=89bf34b8d13d4dea8257337bbdd45ace&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=1953445516ad4bdf957eed35d5c68504 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:38.135217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086b987-aedc-43e6-8c20-6a319d5e6d1a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.135217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086b987-aedc-43e6-8c20-6a319d5e6d1a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.135217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086b987-aedc-43e6-8c20-6a319d5e6d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.154539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7086b987-aedc-43e6-8c20-6a319d5e6d1a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:38.156566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7086b987-aedc-43e6-8c20-6a319d5e6d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:38.185947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:38.244779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332c391-9df6-4bbe-8886-413f8cb6feca None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.259929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 64/120] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:38.264802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086b987-aedc-43e6-8c20-6a319d5e6d1a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.312416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.312976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.313530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.313675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.314303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:38.314641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:38.318158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:38.327610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 57/121] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:38.328563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.397331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086b987-aedc-43e6-8c20-6a319d5e6d1a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.399152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:38.434307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.435344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.435914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.436449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.450448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:38.459074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:38.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:38.462103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:38.462862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:38.464121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 65/122] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:38.465853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.481231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249032-70ca-4b75-a843-517d2abfb928 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.548504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.549224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.549507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.549794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.552470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:38.555074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:38.559491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 58/123] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:38.560563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.631607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071287e-f991-4613-9b02-9b595f9ae9b4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.667376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.668032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.668321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.668607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] PATH_INFO: `/v3/users/66932a6f9a244fcaa7375a468f28c399` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.669761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] RBAC: Authorizing `identity:get_user(user_id=66932a6f9a244fcaa7375a468f28c399)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:38.671386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:38.673031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 66/124] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/users/66932a6f9a244fcaa7375a468f28c399 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:38.673823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.715004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bc8b1-adef-4541-8779-faee157ba5e0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.731331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:38.753160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:38.758585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.759501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.759988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.760437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.761289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:38.761769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:38.763375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:38.777815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 59/125] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:38.779006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.793232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b302f-5ba1-4d99-b00e-b63ff62c3f4e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.829608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.830284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.830572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.830855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.832057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:38.833534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:38.835226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 67/126] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:38.841353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:38.909128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bdfa8-67d9-418b-9ba2-f2f4eb312f34 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.951962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:38.952796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:38.953201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:38.953560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:38.957820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3762-8178-4b9e-a3aa-378a9d4e84a1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:38.964476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:38.970602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:38.971716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:38.972693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:38.974614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 60/127] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:38.975629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.006060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.007303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.007750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.008459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.010456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.013015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.015218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 68/128] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:39.016428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.017988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:39.022382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:39.103175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be31e3-09a3-4bb3-a58b-b596bf8e494b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:39.149662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b085cb4-873e-4d06-ae78-45beb90a0f17 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:39.153024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.153668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.153961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.154239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.158960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.161377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.164663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 61/129] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:38 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:39.165733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.201413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.202741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.203047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.203613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/636cfdf20b5042e8a116abd580b78f06/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.206310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] RBAC: Authorizing `identity:create_grant(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9, user_id=636cfdf20b5042e8a116abd580b78f06, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.208105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.234672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 69/130] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:39 2022] PUT /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/636cfdf20b5042e8a116abd580b78f06/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:39.239432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.337945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5142b-a8f6-4ded-b118-d5def31d3cec None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:39.351233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:39.377803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:39.419129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.419759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.420047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.420395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.421139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:39.421460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.424323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.443903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 62/131] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:39.446169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.468393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536c869-171d-44ee-b86d-a09a6acbd201 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:39.510511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.511276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.511558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.511895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/66932a6f9a244fcaa7375a468f28c399/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.513995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] RBAC: Authorizing `identity:create_grant(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9, user_id=66932a6f9a244fcaa7375a468f28c399, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.515819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.534412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 70/132] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:39 2022] PUT /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/66932a6f9a244fcaa7375a468f28c399/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:39.535408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.592381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2476a-5bdd-46fa-a6a6-4b4bec3ba280 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:39.661926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.664126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.665324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.665787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.667972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.672656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.674740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 63/133] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:39 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:39.684267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.761056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e44be-9935-4a8d-8762-b4299cb24851 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:39.778003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:39.816503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:39.860286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d6b1-49f8-425a-9d01-5de19d27e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.860974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d6b1-49f8-425a-9d01-5de19d27e911 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.861270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d6b1-49f8-425a-9d01-5de19d27e911 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.861538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d6b1-49f8-425a-9d01-5de19d27e911 None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.865216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d6b1-49f8-425a-9d01-5de19d27e911 None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.866716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d6b1-49f8-425a-9d01-5de19d27e911 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.868379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 71/134] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:39 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:39.869771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ae798-e569-4f71-b684-ddc697b99bc0 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.869987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ae798-e569-4f71-b684-ddc697b99bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.870278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ae798-e569-4f71-b684-ddc697b99bc0 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.871811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 72/135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:39.877517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ae798-e569-4f71-b684-ddc697b99bc0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:39.878215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffd278-88bf-46f6-9191-0ea490ac1989 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:39.914313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4ebfe-d24c-4a73-b699-fa784b12781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:39.915628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ebfe-d24c-4a73-b699-fa784b12781f None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.916188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ebfe-d24c-4a73-b699-fa784b12781f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.916667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ebfe-d24c-4a73-b699-fa784b12781f None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/636cfdf20b5042e8a116abd580b78f06/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.919844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ebfe-d24c-4a73-b699-fa784b12781f None admin] RBAC: Authorizing `identity:create_grant(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9, user_id=636cfdf20b5042e8a116abd580b78f06, role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:39.921647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ebfe-d24c-4a73-b699-fa784b12781f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:39.939523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 64/136] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:39 2022] PUT /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/636cfdf20b5042e8a116abd580b78f06/roles/c37a4d448aa147ebb487e7de2099e1db => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:39.942102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbef16-f75a-45f6-ad9d-5556372feb05 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:39.942614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbef16-f75a-45f6-ad9d-5556372feb05 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:39.943435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbef16-f75a-45f6-ad9d-5556372feb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:39.971680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e7dbef16-f75a-45f6-ad9d-5556372feb05 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:39.973533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e7dbef16-f75a-45f6-ad9d-5556372feb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:39.997613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ae798-e569-4f71-b684-ddc697b99bc0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:40.037545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:40.038498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:40.038884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:40.039408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] PATH_INFO: `/v3/projects/f221fd0545b14ea1adf232f3dfccadda` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:40.040648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] RBAC: Authorizing `identity:get_project(project_id=f221fd0545b14ea1adf232f3dfccadda)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:40.042345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:40.044374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 73/137] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:39 2022] GET /identity/v3/projects/f221fd0545b14ea1adf232f3dfccadda => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:40.052720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:40.065477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 65/138] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:40.124831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:40.169146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2c2d9-7286-45d1-b736-f94bf960a726 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:40.206547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d39ce-ad81-4bcd-bba7-51019d0b0866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:40.207529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d39ce-ad81-4bcd-bba7-51019d0b0866 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:40.207982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d39ce-ad81-4bcd-bba7-51019d0b0866 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:40.208403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d39ce-ad81-4bcd-bba7-51019d0b0866 None admin] PATH_INFO: `/v3/projects/f221fd0545b14ea1adf232f3dfccadda/users/636cfdf20b5042e8a116abd580b78f06/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:40.210540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d39ce-ad81-4bcd-bba7-51019d0b0866 None admin] RBAC: Authorizing `identity:create_grant(project_id=f221fd0545b14ea1adf232f3dfccadda, user_id=636cfdf20b5042e8a116abd580b78f06, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:40.212264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d39ce-ad81-4bcd-bba7-51019d0b0866 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:40.229222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 74/139] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:40 2022] PUT /identity/v3/projects/f221fd0545b14ea1adf232f3dfccadda/users/636cfdf20b5042e8a116abd580b78f06/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:40.484645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbef16-f75a-45f6-ad9d-5556372feb05 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:40.517756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:40.535430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a61d-fa1f-4e50-aa29-509260d43533 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:40.535726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a61d-fa1f-4e50-aa29-509260d43533 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:40.536109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a61d-fa1f-4e50-aa29-509260d43533 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:40.537473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 75/140] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:29:40.543932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49884ee-3d26-419c-96fc-d3d0651dabd1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:40.544338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49884ee-3d26-419c-96fc-d3d0651dabd1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:40.544534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49884ee-3d26-419c-96fc-d3d0651dabd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:40.577668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c49884ee-3d26-419c-96fc-d3d0651dabd1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:40.581427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c49884ee-3d26-419c-96fc-d3d0651dabd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:40.685994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 76/141] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:40.722894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbef16-f75a-45f6-ad9d-5556372feb05 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:40.789224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:40.789885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:40.790171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:40.790452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:40.791615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:40.793152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:40.795068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 66/142] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:40.800424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49884ee-3d26-419c-96fc-d3d0651dabd1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:40.883682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:40.929023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49884ee-3d26-419c-96fc-d3d0651dabd1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:40.965518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:40.966144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:40.966427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:40.966705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:40.967434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:40.968922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:40.998912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 77/143] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:29:40 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:41.221251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:41.317625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:41.326489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde3ecd-99db-4a0f-862b-92559153fd1d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:41.361436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:41.362086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:41.362372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:41.362706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:41.373339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:41.378184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:41.379062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:41.379910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 67/144] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:41.385219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:41.492034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45171bf9-2bb3-47bb-b745-6068867ed3af None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:41.499929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:41.528030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:41.529219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:41.529924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:41.530207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:41.530911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:41.532802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:41.532802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:41.544594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 78/145] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:41.550663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:41.672738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f9e7b-c392-47c5-a955-2a26f6cbf94f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:41.714237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:41.715155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:41.715688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:41.716111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] PATH_INFO: `/v3/users/89bf34b8d13d4dea8257337bbdd45ace` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:41.717900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] RBAC: Authorizing `identity:get_user(user_id=89bf34b8d13d4dea8257337bbdd45ace)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:41.719592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:41.721411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 68/146] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:41 2022] GET /identity/v3/users/89bf34b8d13d4dea8257337bbdd45ace => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:41.727327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:41.865347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a474e9-ed51-48ea-aa22-5b8e16e2a99c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:41.907958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51f45-4d6a-4831-a879-6b7ad2c89ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:41.908628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51f45-4d6a-4831-a879-6b7ad2c89ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:41.908919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51f45-4d6a-4831-a879-6b7ad2c89ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:41.909208 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51f45-4d6a-4831-a879-6b7ad2c89ff1 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:41.910468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df51f45-4d6a-4831-a879-6b7ad2c89ff1 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:41.911988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df51f45-4d6a-4831-a879-6b7ad2c89ff1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:41.914146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 79/147] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:41 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:41.919890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:41.956860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:42.050670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f0c6-42a6-4c09-9143-f1bc0bd3f9ef None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:42.095881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca5017-addf-496d-9be4-46477fce7a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:42.097024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5017-addf-496d-9be4-46477fce7a5c None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:42.097530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5017-addf-496d-9be4-46477fce7a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:42.098012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5017-addf-496d-9be4-46477fce7a5c None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/89bf34b8d13d4dea8257337bbdd45ace/roles/1953445516ad4bdf957eed35d5c68504` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:42.100964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca5017-addf-496d-9be4-46477fce7a5c None admin] RBAC: Authorizing `identity:create_grant(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9, user_id=89bf34b8d13d4dea8257337bbdd45ace, role_id=1953445516ad4bdf957eed35d5c68504)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:42.102666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca5017-addf-496d-9be4-46477fce7a5c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:42.118021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 69/148] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:41 2022] PUT /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/users/89bf34b8d13d4dea8257337bbdd45ace/roles/1953445516ad4bdf957eed35d5c68504 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:42.247051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a915bd9-8d13-4e2a-96d0-6bb810964774 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:42.247604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47aa8-59e8-4e95-a805-5bf3e3ffd47c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:42.247681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a915bd9-8d13-4e2a-96d0-6bb810964774 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:42.247748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47aa8-59e8-4e95-a805-5bf3e3ffd47c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:42.247847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a915bd9-8d13-4e2a-96d0-6bb810964774 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:42.247917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47aa8-59e8-4e95-a805-5bf3e3ffd47c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:42.249621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 80/150] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:42.249725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 70/150] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:29:42.255714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4cc0b-b6d4-4830-b7b4-af1caa30d4f0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:42.255992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4cc0b-b6d4-4830-b7b4-af1caa30d4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:42.256272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4cc0b-b6d4-4830-b7b4-af1caa30d4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:42.260605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788da07-dc2d-4c9d-9c6c-71ef5aa4dd9f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:42.260894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788da07-dc2d-4c9d-9c6c-71ef5aa4dd9f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:42.261172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788da07-dc2d-4c9d-9c6c-71ef5aa4dd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:42.301246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4788da07-dc2d-4c9d-9c6c-71ef5aa4dd9f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:42.302836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-92d4cc0b-b6d4-4830-b7b4-af1caa30d4f0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:42.304014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-92d4cc0b-b6d4-4830-b7b4-af1caa30d4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:42.306315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4788da07-dc2d-4c9d-9c6c-71ef5aa4dd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:42.376822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:42.434348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 81/151] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:42.458181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 71/152] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:42.884430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5530344-1f5f-4c04-a092-6c8adfafdf68 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:42.884805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5530344-1f5f-4c04-a092-6c8adfafdf68 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:42.884989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5530344-1f5f-4c04-a092-6c8adfafdf68 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:42.886551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 82/153] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:42.898325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ad89-64b1-4857-a72b-e0ea1a7ed572 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:42.898854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ad89-64b1-4857-a72b-e0ea1a7ed572 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:42.898854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ad89-64b1-4857-a72b-e0ea1a7ed572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:42.924065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ee91ad89-64b1-4857-a72b-e0ea1a7ed572 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:42.925267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ee91ad89-64b1-4857-a72b-e0ea1a7ed572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:42.932401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:43.005500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5530344-1f5f-4c04-a092-6c8adfafdf68 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:43.007592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 72/154] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:43.023071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91ad89-64b1-4857-a72b-e0ea1a7ed572 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:43.220720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5530344-1f5f-4c04-a092-6c8adfafdf68 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:43.224678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91ad89-64b1-4857-a72b-e0ea1a7ed572 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:43.290800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:43.292589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:43.293384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:43.294467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:43.295474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:43.296420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:43.297470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:43.297940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:43.314476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:43.320018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:43.320747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:43.329709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:43.330633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:43.331459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:43.334682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:43.335755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:43.336873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:43.343537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 73/155] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:43.344685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 83/156] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:43.346687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:43.350939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:43.400773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:43.520254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4f41-0118-4fc3-930e-443db9e2de6b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:43.566339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8d08-4c6a-4297-92ae-158b99e465fc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:43.596415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe6d11-2a14-40bf-953d-0237cf42be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:43.597051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe6d11-2a14-40bf-953d-0237cf42be73 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:43.597637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe6d11-2a14-40bf-953d-0237cf42be73 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:43.598109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe6d11-2a14-40bf-953d-0237cf42be73 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:43.599361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe6d11-2a14-40bf-953d-0237cf42be73 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:43.599796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe6d11-2a14-40bf-953d-0237cf42be73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:43.601227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe6d11-2a14-40bf-953d-0237cf42be73 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:43.612952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 84/157] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:43.614289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c99bb-c831-495e-a498-b8f39da65bba None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:43.614581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c99bb-c831-495e-a498-b8f39da65bba None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:43.614858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c99bb-c831-495e-a498-b8f39da65bba None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:43.616258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 85/158] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:43.616950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c99bb-c831-495e-a498-b8f39da65bba None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:43.633800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa54558-b550-4a73-9f4f-350162409399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:43.634785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54558-b550-4a73-9f4f-350162409399 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:43.635297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54558-b550-4a73-9f4f-350162409399 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:43.635771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa54558-b550-4a73-9f4f-350162409399 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:43.636674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa54558-b550-4a73-9f4f-350162409399 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:43.637174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa54558-b550-4a73-9f4f-350162409399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:43.647219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa54558-b550-4a73-9f4f-350162409399 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:43.661471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 74/159] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:43.663059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f4cfe-589a-4776-af7f-4e3205d5ad0e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:43.663612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f4cfe-589a-4776-af7f-4e3205d5ad0e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:43.664087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f4cfe-589a-4776-af7f-4e3205d5ad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:43.693286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fc0f4cfe-589a-4776-af7f-4e3205d5ad0e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:43.694830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fc0f4cfe-589a-4776-af7f-4e3205d5ad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:43.777376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:43.805812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:43.806783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c99bb-c831-495e-a498-b8f39da65bba None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:43.833799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 75/160] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:43.834437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f4cfe-589a-4776-af7f-4e3205d5ad0e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:43.847640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:43.848340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:43.848673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:43.848951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:43.861081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:43.866022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:43.866918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:43.867753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:43.871630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 86/161] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:29:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:43.872481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:43.965908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f4cfe-589a-4776-af7f-4e3205d5ad0e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:44.002444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcb3b-446e-4d84-a17b-253a90c705df None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:44.039622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:44.040239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:44.040734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:44.041020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] PATH_INFO: `/v3/users/66932a6f9a244fcaa7375a468f28c399` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:44.042171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=66932a6f9a244fcaa7375a468f28c399)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:44.043743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:44.045451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 87/162] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:43 2022] GET /identity/v3/users/66932a6f9a244fcaa7375a468f28c399 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:44.046422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:44.079882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:44.112049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:44.112731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:44.113024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:44.113339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:44.119999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:44.121509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:44.123650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 76/163] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:43 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:44.124635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:44.167971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f742809-1f08-4a04-9b53-a81ace974cd6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:44.206634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:44.207590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:44.207882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:44.208561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:44.217993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:44.218496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:44.220381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:44.229718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 88/164] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:29:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:44.230553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:44.285160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:44.356632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b7a7f-586f-44b2-9438-ed90278b5920 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:44.384691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:44.411636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea8b80-fc53-4ede-88dc-6a01c60d4852 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:44.433905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:44.434542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:44.434900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:44.435270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:44.488109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:44.488995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:44.489288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:44.489596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:44.569987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:44.571514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:44.617165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:44.619620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:44.704537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:44.713113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:44.745497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 89/165] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:44.746479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:44.788399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 77/166] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:44.789273 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:44.954274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56323a0-506d-4945-be31-72cc8f3ae5da None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:44.986541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0ec6-bafc-4447-a4cf-686fd3aa5c90 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:45.025279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.026035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.026315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.026593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.027790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:45.027948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.028729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.029207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.029321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:45.029745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.031668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 90/167] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:44 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:45.036779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.048412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:45.054011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:45.055169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:45.056310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:45.058574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 78/168] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:45.059781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.104491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:45.106846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:45.163223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf327c2-f433-469b-b762-854f70d9bc5e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:45.202372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d930c6d-2c26-4fcc-9a3c-759def1c3a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.202372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d930c6d-2c26-4fcc-9a3c-759def1c3a4e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.202372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d930c6d-2c26-4fcc-9a3c-759def1c3a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.202372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d930c6d-2c26-4fcc-9a3c-759def1c3a4e None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.203656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d930c6d-2c26-4fcc-9a3c-759def1c3a4e None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:45.205393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d930c6d-2c26-4fcc-9a3c-759def1c3a4e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:45.207219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 91/169] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:45.215217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a38edd-d135-455b-9c67-c61e7f877d3e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.215217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a38edd-d135-455b-9c67-c61e7f877d3e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.215217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a38edd-d135-455b-9c67-c61e7f877d3e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.216345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 92/170] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:45.217341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a38edd-d135-455b-9c67-c61e7f877d3e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.333103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a38edd-d135-455b-9c67-c61e7f877d3e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:45.341556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f78f-2245-44e7-a859-473497796824 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:45.359588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:45.379728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.380763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.381249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.381724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.383089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:45.384956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:45.386825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 79/171] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:45.387812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.439785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.440482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.440697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.440972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.482419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:45.495093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d5fcf-c7f5-4175-8ebc-8113592c8451 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:45.522850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:45.524649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:45.532470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bca2ae-ae9f-4713-948a-e70bc742961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.533178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca2ae-ae9f-4713-948a-e70bc742961b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.533527 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca2ae-ae9f-4713-948a-e70bc742961b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.533865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca2ae-ae9f-4713-948a-e70bc742961b None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.534612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca2ae-ae9f-4713-948a-e70bc742961b None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:45.534987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca2ae-ae9f-4713-948a-e70bc742961b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:45.536504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca2ae-ae9f-4713-948a-e70bc742961b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:45.545633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 80/172] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:45.547766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b52666-6df8-46ef-b3a5-b229c4b9a941 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.548155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b52666-6df8-46ef-b3a5-b229c4b9a941 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.548532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b52666-6df8-46ef-b3a5-b229c4b9a941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.570714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b6b52666-6df8-46ef-b3a5-b229c4b9a941 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:45.572101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b6b52666-6df8-46ef-b3a5-b229c4b9a941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:45.632253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 93/173] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:45.633232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.648234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 81/174] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:45.649312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b52666-6df8-46ef-b3a5-b229c4b9a941 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.649865 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:45.744021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:45.783568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b52666-6df8-46ef-b3a5-b229c4b9a941 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:45.812730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86969d8b-7389-43a8-803f-093fdc9d31dc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:45.827652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd92437-1888-4225-882d-4a1a59669752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.829376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd92437-1888-4225-882d-4a1a59669752 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.831602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd92437-1888-4225-882d-4a1a59669752 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.832236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd92437-1888-4225-882d-4a1a59669752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.834722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd92437-1888-4225-882d-4a1a59669752 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=66932a6f9a244fcaa7375a468f28c399) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:45.835441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd92437-1888-4225-882d-4a1a59669752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:45.836956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd92437-1888-4225-882d-4a1a59669752 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:45.849655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 82/175] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/role_assignments?user.id=66932a6f9a244fcaa7375a468f28c399&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:45.852477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff694a-5b8d-4fd5-abdd-93c612a0b357 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.853037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff694a-5b8d-4fd5-abdd-93c612a0b357 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.853409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff694a-5b8d-4fd5-abdd-93c612a0b357 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.855115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 83/176] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:45.856011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff694a-5b8d-4fd5-abdd-93c612a0b357 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.857937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:45.859167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:45.859716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:45.860200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:45.861703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=636cfdf20b5042e8a116abd580b78f06) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:45.863161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:45.865031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:45.873194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 94/177] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/role_assignments?user.id=636cfdf20b5042e8a116abd580b78f06&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:45.875179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:45.982063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:46.000182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff694a-5b8d-4fd5-abdd-93c612a0b357 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.014966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceed250-fc84-4ead-8979-060fdbd7cf08 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.058755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de21aaa-5059-4a53-887c-22031fba3377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:46.059631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21aaa-5059-4a53-887c-22031fba3377 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.059867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21aaa-5059-4a53-887c-22031fba3377 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.060146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21aaa-5059-4a53-887c-22031fba3377 None admin] PATH_INFO: `/v3/users/636cfdf20b5042e8a116abd580b78f06` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.063730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de21aaa-5059-4a53-887c-22031fba3377 None admin] RBAC: Authorizing `identity:get_user(user_id=636cfdf20b5042e8a116abd580b78f06)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:46.065229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de21aaa-5059-4a53-887c-22031fba3377 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:46.069043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 84/178] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/users/636cfdf20b5042e8a116abd580b78f06 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:46.069043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3304e37-326e-4836-98f4-2741fd0fcd89 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.069043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3304e37-326e-4836-98f4-2741fd0fcd89 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.069043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3304e37-326e-4836-98f4-2741fd0fcd89 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.070121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 85/179] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:46.074060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c201d-649a-416b-80c7-b2b99981fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:46.074984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c201d-649a-416b-80c7-b2b99981fd0c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.075447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c201d-649a-416b-80c7-b2b99981fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.075950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c201d-649a-416b-80c7-b2b99981fd0c None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.076822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636e935-e176-40aa-86bc-ae33ae5e82b1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.077250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636e935-e176-40aa-86bc-ae33ae5e82b1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.077531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636e935-e176-40aa-86bc-ae33ae5e82b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.078175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c201d-649a-416b-80c7-b2b99981fd0c None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:46.081206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c201d-649a-416b-80c7-b2b99981fd0c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:46.083268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 95/180] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:45 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:46.084642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6d29-6e41-42d3-8def-7119c4dc2fe7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.090154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6d29-6e41-42d3-8def-7119c4dc2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.090689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae6d29-6e41-42d3-8def-7119c4dc2fe7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.092559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 96/181] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:46.093538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae6d29-6e41-42d3-8def-7119c4dc2fe7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:46.106357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1636e935-e176-40aa-86bc-ae33ae5e82b1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:46.107548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1636e935-e176-40aa-86bc-ae33ae5e82b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:46.117012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:46.220426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 86/182] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:46.222151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad629c-c4a0-4f38-83c5-3940a0b99b88 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.222446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad629c-c4a0-4f38-83c5-3940a0b99b88 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.222729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad629c-c4a0-4f38-83c5-3940a0b99b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.243677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae6d29-6e41-42d3-8def-7119c4dc2fe7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.245204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dfad629c-c4a0-4f38-83c5-3940a0b99b88 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:46.247714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dfad629c-c4a0-4f38-83c5-3940a0b99b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:46.280526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a21c4-2a99-4367-b4e2-9dcd3b5ff7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:46.281437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a21c4-2a99-4367-b4e2-9dcd3b5ff7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.281723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a21c4-2a99-4367-b4e2-9dcd3b5ff7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.282010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a21c4-2a99-4367-b4e2-9dcd3b5ff7f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.299662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:46.343237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 87/183] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:46.343237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad629c-c4a0-4f38-83c5-3940a0b99b88 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:46.363213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a21c4-2a99-4367-b4e2-9dcd3b5ff7f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:46.365058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a21c4-2a99-4367-b4e2-9dcd3b5ff7f9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:46.470482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:46.485798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 97/184] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:46.487158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd88ad-082b-4f89-943c-5ec68776be29 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.487427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd88ad-082b-4f89-943c-5ec68776be29 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.487725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd88ad-082b-4f89-943c-5ec68776be29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.507707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b0fd88ad-082b-4f89-943c-5ec68776be29 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:46.508485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b0fd88ad-082b-4f89-943c-5ec68776be29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:46.510491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad629c-c4a0-4f38-83c5-3940a0b99b88 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.554395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:46.556464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:46.559298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.559298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.559298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.559298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] RBAC: Adding query filter params (role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=636cfdf20b5042e8a116abd580b78f06) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:46.559298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:46.564000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:46.570876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 88/185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:46 2022] GET /identity/v3/role_assignments?user.id=636cfdf20b5042e8a116abd580b78f06&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:46.579547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:46.585395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 98/186] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:46.586971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd88ad-082b-4f89-943c-5ec68776be29 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:46.743394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd88ad-082b-4f89-943c-5ec68776be29 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.751886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e98eb-10ce-4a72-9cea-1e13ebbe5b16 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.799710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:46.800555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.800889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.801169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.804524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:46.804923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:46.806427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.806992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.807542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] PATH_INFO: `/v3/projects/f221fd0545b14ea1adf232f3dfccadda` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:46.810539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] RBAC: Authorizing `identity:get_project(project_id=f221fd0545b14ea1adf232f3dfccadda)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:46.812190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:46.814400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 99/187] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:46 2022] GET /identity/v3/projects/f221fd0545b14ea1adf232f3dfccadda => generated 312 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:46.815412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:46.823238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:46.831306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:46.832258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:46.833102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 89/188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:46.833788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:46.882453 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:46.949574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4afd9f-73f8-4c91-a84a-1ba053c75548 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.973520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066f76-7813-4ffc-8f76-5550bcd00037 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:46.987850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:46.988495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:46.988782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:46.989059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.000919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:47.005799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:47.006739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:47.007699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 100/189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:47.009333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.029409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.030279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.030628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.030982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.032229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.033865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.035578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 90/190] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:47.036952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.103709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.148028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f0c58-4fe0-42c8-93e0-744a26cd4fad None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.148521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230553b-5779-4910-87ec-f6684d145b3c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.150933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.182572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.183186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.183502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.183775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.184451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:47.184540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.184758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.185126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.185409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.185684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.186138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.186451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=f221fd0545b14ea1adf232f3dfccadda, user.id=636cfdf20b5042e8a116abd580b78f06) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:47.186762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.188606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.194728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 91/192] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:47.194882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 101/192] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/role_assignments?user.id=636cfdf20b5042e8a116abd580b78f06&scope.project.id=f221fd0545b14ea1adf232f3dfccadda&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:47.195643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.195765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.308503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd819995-6a0b-43bf-95a5-16ae62770e04 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.319152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d39fd0-1321-4287-a972-7b165c346c8f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.338886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.348961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.349604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.349885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.350158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.363726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.363952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:47.364294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.364576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.364848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.365527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:47.365830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.367248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:47.369550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:47.370450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 92/193] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:47.375840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.387300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 102/194] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:47.387300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.417739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.491571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0951bf9-1bea-4ccb-badb-d0620d986b9f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.492394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb964d50-30b3-48ec-a64c-00674f5177b3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.525277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.525894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.526036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.526177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.526452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.526905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.527113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.527242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.527494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] PATH_INFO: `/v3/users/5809cfedddbb40efbc2c2fa52896d930` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.528606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.543302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] RBAC: Authorizing `identity:get_user(user_id=5809cfedddbb40efbc2c2fa52896d930)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.544840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.547062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 103/195] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/users/5809cfedddbb40efbc2c2fa52896d930 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:47.547806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.554466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 93/196] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:29:47 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:47.555314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.609753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.656929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.673628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bf8c9-f18c-4f48-ba0e-d26148fe336a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.682376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e52acc-d53f-477d-996c-6c3f97a83474 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.715746 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.716367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.716651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.716925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.717702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:47.717944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.720146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.726254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.726561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.726846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.727644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] PATH_INFO: `/v3/users/89bf34b8d13d4dea8257337bbdd45ace` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.728753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] RBAC: Authorizing `identity:get_user(user_id=89bf34b8d13d4dea8257337bbdd45ace)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.730221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.732129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 94/197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/users/89bf34b8d13d4dea8257337bbdd45ace => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:47.736493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 104/198] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:47.736622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.737747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:47.860089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95530568-dc88-4a6b-bafb-262669b66d1b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.865286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9722-c250-443c-bd27-51123efd3407 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:47.872186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.905663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.906396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.906673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.907004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.918234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:47.919670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:47.920214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:47.921151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:47.959627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:47.980178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.981810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:47.994095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:47.995945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.093577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 95/199] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:48.094595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:48.107317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:48.120253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 105/200] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:48.121032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:48.189839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:48.211975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50168cc4-3ca3-4314-8b69-ac3bc32c8f21 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:48.253688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:48.254265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f0aec-9bb9-411c-ad0e-de13f7c4990d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:48.254877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:48.255352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:48.255867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] PATH_INFO: `/v3/users/5809cfedddbb40efbc2c2fa52896d930` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:48.257906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] RBAC: Authorizing `identity:get_user(user_id=5809cfedddbb40efbc2c2fa52896d930)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:48.259582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.261368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 96/201] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/users/5809cfedddbb40efbc2c2fa52896d930 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:48.262454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:48.298658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:48.299657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:48.300228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:48.300513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:48.301654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:48.303380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.305917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 106/202] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:48.306695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:48.352298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:48.378048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bef05-bf06-4941-8534-6bff5bac9708 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:48.413855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:48.414504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:48.414788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:48.415066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:48.416242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:48.417713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.419940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 97/203] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:48.420397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:48.434846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865d428-e9eb-4cf1-9a94-5320307b7f23 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:48.458842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:48.481420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:48.482582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:48.483615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:48.484372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:48.537013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00b018-212d-4aa0-9b9b-d34bd66eb23f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:48.559840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:48.561506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.585233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:48.586453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:48.586875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:48.587571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:48.588923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=5809cfedddbb40efbc2c2fa52896d930) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:48.589394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:48.591763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.599756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 98/204] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/role_assignments?user.id=5809cfedddbb40efbc2c2fa52896d930&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:48.600592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:48.619506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:48.739950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 107/205] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:48.745916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:48.765858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:48.769269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70f7e0-292f-4891-b2d5-f9334eddf198 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:48.816471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:48.817094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:48.817390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:48.817975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:48.819001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] RBAC: Adding query filter params (role.id=1953445516ad4bdf957eed35d5c68504, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9, user.id=89bf34b8d13d4dea8257337bbdd45ace) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:48.819350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:48.821027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.829446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 99/206] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/role_assignments?user.id=89bf34b8d13d4dea8257337bbdd45ace&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=1953445516ad4bdf957eed35d5c68504 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:48.873026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4fe9f-eda3-4915-a67c-fdb20d4bdc62 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:48.921065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f665c2-ffff-4de4-9519-1b9ba4f477cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:48.921734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f665c2-ffff-4de4-9519-1b9ba4f477cc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:48.922019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f665c2-ffff-4de4-9519-1b9ba4f477cc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:48.922298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f665c2-ffff-4de4-9519-1b9ba4f477cc None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:48.928073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f665c2-ffff-4de4-9519-1b9ba4f477cc None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:48.929611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f665c2-ffff-4de4-9519-1b9ba4f477cc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:48.931506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 108/207] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:48.938407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:49.038886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:49.065706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fbdcc-f344-4a1d-91cf-63b23511febb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:49.101355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7579ef-b960-4304-9796-993692b99c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:49.101984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7579ef-b960-4304-9796-993692b99c60 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:49.102240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7579ef-b960-4304-9796-993692b99c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:49.102516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7579ef-b960-4304-9796-993692b99c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:49.103282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7579ef-b960-4304-9796-993692b99c60 None admin] RBAC: Adding query filter params (role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=5809cfedddbb40efbc2c2fa52896d930) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:49.103594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7579ef-b960-4304-9796-993692b99c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:49.105035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7579ef-b960-4304-9796-993692b99c60 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:49.111526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 100/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:48 2022] GET /identity/v3/role_assignments?user.id=5809cfedddbb40efbc2c2fa52896d930&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:49.756531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26de245-6c65-486d-b8a5-03bb4d3c0439 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:49.757063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26de245-6c65-486d-b8a5-03bb4d3c0439 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:49.757063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26de245-6c65-486d-b8a5-03bb4d3c0439 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:49.758698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 109/209] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:49.761067 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316c538-5a95-46a0-965e-f4947c798cef None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:49.761365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316c538-5a95-46a0-965e-f4947c798cef None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:49.761650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316c538-5a95-46a0-965e-f4947c798cef None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:49.763107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 101/210] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:49.766769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4f0b-d923-46f5-a6a1-736c163dcf64 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:49.767316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4f0b-d923-46f5-a6a1-736c163dcf64 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:49.767985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4f0b-d923-46f5-a6a1-736c163dcf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:49.768886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf83b6-a303-4d12-b09a-3a735a33ed04 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:49.769174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf83b6-a303-4d12-b09a-3a735a33ed04 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:49.769457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf83b6-a303-4d12-b09a-3a735a33ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:49.789468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-87bf83b6-a303-4d12-b09a-3a735a33ed04 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:49.791024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c69c4f0b-d923-46f5-a6a1-736c163dcf64 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:49.791383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-87bf83b6-a303-4d12-b09a-3a735a33ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:49.792266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c69c4f0b-d923-46f5-a6a1-736c163dcf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:49.869444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 102/211] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:49.878209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 110/212] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:50.134364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf83b6-a303-4d12-b09a-3a735a33ed04 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:50.152101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4f0b-d923-46f5-a6a1-736c163dcf64 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:50.178300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:50.206607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:50.257339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf83b6-a303-4d12-b09a-3a735a33ed04 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:50.289604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4f0b-d923-46f5-a6a1-736c163dcf64 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:50.298775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:50.299679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.300180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.300596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.313137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:50.318370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:50.319307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:50.320298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:50.327227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 103/213] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:50.328376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:50.331737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:50.332517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.332798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.333864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.335010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:50.336497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:50.338490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 111/214] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:50.344350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:50.429280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:50.441273 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cad1ac-1588-4225-9374-aa195a8879ac None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:50.446807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ead0f-caf1-4705-a2d6-08765c2d3b20 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:50.471064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:50.481040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:50.481843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.482269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.482685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.482946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:50.483643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.483918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:50.484029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.484341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.484660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:50.485001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:50.486345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:50.486460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:50.495770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 104/215] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:50.503134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:50.515739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 112/216] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:29:50 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:29:50.625282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee74b1-4c5b-44a0-aae4-a5d2c68980fe None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:50.665470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888855b-d119-4a6f-a31c-54ae81d0ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:50.666434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888855b-d119-4a6f-a31c-54ae81d0ca9b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.666963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888855b-d119-4a6f-a31c-54ae81d0ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.667464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888855b-d119-4a6f-a31c-54ae81d0ca9b None admin] PATH_INFO: `/v3/users/f89e85bacb4e422d9af96269f2347e54` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.682679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888855b-d119-4a6f-a31c-54ae81d0ca9b None admin] RBAC: Authorizing `identity:get_user(user_id=f89e85bacb4e422d9af96269f2347e54)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:50.684156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888855b-d119-4a6f-a31c-54ae81d0ca9b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:50.686081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 105/217] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:50 2022] GET /identity/v3/users/f89e85bacb4e422d9af96269f2347e54 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:50.692175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:50.776775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5306-3711-4d07-b738-ce28696a95e3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.777238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5306-3711-4d07-b738-ce28696a95e3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.777395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5306-3711-4d07-b738-ce28696a95e3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.778735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 106/218] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:29:50.785910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe40db-06ac-4be1-8438-4a9cd357344c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.786204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe40db-06ac-4be1-8438-4a9cd357344c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.786469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe40db-06ac-4be1-8438-4a9cd357344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.807843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-60fe40db-06ac-4be1-8438-4a9cd357344c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:50.809366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-60fe40db-06ac-4be1-8438-4a9cd357344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:50.821378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489fce-88ca-493c-a2a3-999efd52a157 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:50.868468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70203e-3cda-4686-aa0f-4f41640c3bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:50.869095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70203e-3cda-4686-aa0f-4f41640c3bff None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:50.869377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70203e-3cda-4686-aa0f-4f41640c3bff None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:50.869645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70203e-3cda-4686-aa0f-4f41640c3bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:50.870666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:50.882539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:50.905967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 107/219] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:50.965650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70203e-3cda-4686-aa0f-4f41640c3bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:50.967793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70203e-3cda-4686-aa0f-4f41640c3bff None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:51.079450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 113/220] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:51.085391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe40db-06ac-4be1-8438-4a9cd357344c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:51.086784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c46f-17b9-4ed6-8a9b-14b1f70fcf8f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.087634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c46f-17b9-4ed6-8a9b-14b1f70fcf8f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.088827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35c46f-17b9-4ed6-8a9b-14b1f70fcf8f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.091487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 114/221] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:29:51.098052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe9f4-a989-43ba-afd3-443711e45a0d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.098353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe9f4-a989-43ba-afd3-443711e45a0d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.098609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe9f4-a989-43ba-afd3-443711e45a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.127833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-33dfe9f4-a989-43ba-afd3-443711e45a0d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:51.129070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-33dfe9f4-a989-43ba-afd3-443711e45a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:51.173090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:51.212364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 115/222] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:51.251744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe9f4-a989-43ba-afd3-443711e45a0d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:51.293402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe40db-06ac-4be1-8438-4a9cd357344c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:51.330295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f38c14-e846-496d-9655-025746f13860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:51.330954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f38c14-e846-496d-9655-025746f13860 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.331250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f38c14-e846-496d-9655-025746f13860 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.331531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f38c14-e846-496d-9655-025746f13860 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.332653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f38c14-e846-496d-9655-025746f13860 None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:51.334113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f38c14-e846-496d-9655-025746f13860 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:51.336174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 108/223] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:51.342026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f38c14-e846-496d-9655-025746f13860 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:51.371591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe9f4-a989-43ba-afd3-443711e45a0d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:51.415280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:51.416362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.417978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.418714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.425841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:51.441901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:51.450105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:51.451021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:51.452242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:51.457235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 116/224] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:51.465033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:51.504992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:51.529467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f38c14-e846-496d-9655-025746f13860 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:51.579146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:51.579789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.580072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.580350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.581060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf1c6a-9c1e-48a6-8438-e2df8b60c48e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:51.581179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=f89e85bacb4e422d9af96269f2347e54) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:51.581449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:51.583248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:51.591133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 109/225] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/role_assignments?user.id=f89e85bacb4e422d9af96269f2347e54&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:51.592102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:51.665177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:51.666192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.666479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.666756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.667495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:51.667807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:51.669266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:51.692306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 117/226] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:51.704254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:51.757618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e0034-01b0-40f1-9d23-a073a25a5aad None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:51.781624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:51.826537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:51.828959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:51.829630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.829913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.830190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.838018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a308ad-aec4-458c-ac95-4025d034544b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:51.864716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:51.870083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:51.870980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:51.871801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:51.876956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:51.877701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:51.878048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:51.878391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] PATH_INFO: `/v3/users/5809cfedddbb40efbc2c2fa52896d930` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:51.879663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 110/227] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:51.879772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] RBAC: Authorizing `identity:get_user(user_id=5809cfedddbb40efbc2c2fa52896d930)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:51.881386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:51.883056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 118/228] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/users/5809cfedddbb40efbc2c2fa52896d930 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:51.885669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:51.889202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:52.010734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923d471-ce1f-4404-b09f-03bf74d6811a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:52.048197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:52.048879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:52.049214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:52.049502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:52.050757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:52.052270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:52.054141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 119/229] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:52.060194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:52.089063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0676ca-ae65-485b-a127-d6948e779d46 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:52.103227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:52.173199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb99051-d778-4ea2-bf15-367563d7f78f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:52.211153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:52.212547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:52.213303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:52.214004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:52.214398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:52.215340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/5809cfedddbb40efbc2c2fa52896d930/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:52.215851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:52.216221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:52.217515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:52.218653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:52.219514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8823c7322d7b4c53ad080968a7bfe121, user_id=5809cfedddbb40efbc2c2fa52896d930, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:52.220922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:52.222468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:52.247397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 111/230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:29:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:52.249639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 120/231] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:52 2022] PUT /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/5809cfedddbb40efbc2c2fa52896d930/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:29:52.256047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:52.336393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:52.430323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8885b-134b-47d4-aade-eefd08361839 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:52.479175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:52.479846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:52.480181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:52.480458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] PATH_INFO: `/v3/users/5809cfedddbb40efbc2c2fa52896d930` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:52.481587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] RBAC: Authorizing `identity:get_user(user_id=5809cfedddbb40efbc2c2fa52896d930)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:52.483290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:52.485088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 112/232] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:52 2022] GET /identity/v3/users/5809cfedddbb40efbc2c2fa52896d930 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:52.490834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:52.590768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:52.600802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac7a5-598c-42ce-8129-6639dded74f4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:52.637711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1edc8-3c31-4ab7-b5a9-2753db3e9779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:52.638366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1edc8-3c31-4ab7-b5a9-2753db3e9779 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:52.638652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1edc8-3c31-4ab7-b5a9-2753db3e9779 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:52.638932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1edc8-3c31-4ab7-b5a9-2753db3e9779 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:52.640281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1edc8-3c31-4ab7-b5a9-2753db3e9779 None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:52.641773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1edc8-3c31-4ab7-b5a9-2753db3e9779 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:52.643655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 121/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:52 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:52.649359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:52.751972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:52.755891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a70f9-a632-45fa-be20-c9682731ce5f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:52.791381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e17c5-adc6-4455-9d23-0d63900ea448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:52.792062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e17c5-adc6-4455-9d23-0d63900ea448 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:52.792350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e17c5-adc6-4455-9d23-0d63900ea448 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:52.792632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e17c5-adc6-4455-9d23-0d63900ea448 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/5809cfedddbb40efbc2c2fa52896d930/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:52.794815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e17c5-adc6-4455-9d23-0d63900ea448 None admin] RBAC: Authorizing `identity:create_grant(project_id=8823c7322d7b4c53ad080968a7bfe121, user_id=5809cfedddbb40efbc2c2fa52896d930, role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:52.796506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e17c5-adc6-4455-9d23-0d63900ea448 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:52.814044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 113/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:52 2022] PUT /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/5809cfedddbb40efbc2c2fa52896d930/roles/c37a4d448aa147ebb487e7de2099e1db => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:53.132199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f3fd4-18ed-4dc8-be11-dbdc4cc1bc79 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:53.132718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f3fd4-18ed-4dc8-be11-dbdc4cc1bc79 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:53.132791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f3fd4-18ed-4dc8-be11-dbdc4cc1bc79 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:53.134393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 122/235] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:53.140068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b195a3-fa45-4481-92c2-a3228bf50da9 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:53.140707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b195a3-fa45-4481-92c2-a3228bf50da9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:53.141189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b195a3-fa45-4481-92c2-a3228bf50da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:53.162936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e07d2b-5e78-42d9-ac92-efb79db86981 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:53.163259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e07d2b-5e78-42d9-ac92-efb79db86981 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:53.163522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e07d2b-5e78-42d9-ac92-efb79db86981 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:53.165163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 123/236] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:53.167402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-60b195a3-fa45-4481-92c2-a3228bf50da9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:53.168780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-60b195a3-fa45-4481-92c2-a3228bf50da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:53.170310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb355-7e8c-4f6e-b88c-829f1cb26e2a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:53.170739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb355-7e8c-4f6e-b88c-829f1cb26e2a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:53.171109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb355-7e8c-4f6e-b88c-829f1cb26e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:53.190101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6bcfb355-7e8c-4f6e-b88c-829f1cb26e2a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:53.191333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6bcfb355-7e8c-4f6e-b88c-829f1cb26e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:53.299395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 124/237] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:53.308584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 114/238] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:53.602365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfb355-7e8c-4f6e-b88c-829f1cb26e2a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:53.695735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b195a3-fa45-4481-92c2-a3228bf50da9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:53.725819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:53.747463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfb355-7e8c-4f6e-b88c-829f1cb26e2a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:53.807608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:53.815758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:53.816414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:53.816698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:53.816977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:53.827416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b195a3-fa45-4481-92c2-a3228bf50da9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:53.842368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:53.847277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:53.848148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:53.849071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 125/239] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:53.855878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:53.895922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:53.896937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:53.897411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:53.897874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:53.903920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:53.905644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:53.907900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 115/240] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:53.912809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:53.971929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abf485-4dea-4910-a727-001514445a3a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:54.010061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a58cc-b7e7-4838-8cf4-3b3fa8991b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:54.011057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a58cc-b7e7-4838-8cf4-3b3fa8991b72 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.011506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a58cc-b7e7-4838-8cf4-3b3fa8991b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.012170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a58cc-b7e7-4838-8cf4-3b3fa8991b72 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.013438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a58cc-b7e7-4838-8cf4-3b3fa8991b72 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:54.013894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a58cc-b7e7-4838-8cf4-3b3fa8991b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:54.015971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a58cc-b7e7-4838-8cf4-3b3fa8991b72 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:54.028473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a4c8c-01ce-4043-920a-7fe2d5b61516 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:54.028632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 126/241] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:54.029881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbfaee-7266-4ced-82f3-bf6542642ba7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.030206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbfaee-7266-4ced-82f3-bf6542642ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.030490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbfaee-7266-4ced-82f3-bf6542642ba7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.033170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 127/242] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:54.033982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbfaee-7266-4ced-82f3-bf6542642ba7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:54.048350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:54.072199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d34b3-366b-45ea-a921-390223a35e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:54.073501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d34b3-366b-45ea-a921-390223a35e69 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.074246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d34b3-366b-45ea-a921-390223a35e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.074670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d34b3-366b-45ea-a921-390223a35e69 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.075770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d34b3-366b-45ea-a921-390223a35e69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:54.078130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d34b3-366b-45ea-a921-390223a35e69 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:54.108250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 116/243] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:29:53 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:29:54.110423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6aae-a38a-498b-8423-9ee91550023b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.110995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6aae-a38a-498b-8423-9ee91550023b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.111516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6aae-a38a-498b-8423-9ee91550023b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.146826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a0ba6aae-a38a-498b-8423-9ee91550023b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:54.146826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a0ba6aae-a38a-498b-8423-9ee91550023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:54.168576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:54.185801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbfaee-7266-4ced-82f3-bf6542642ba7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:54.268031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:54.268758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 117/244] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:54.269217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.269567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.269850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] PATH_INFO: `/v3/users/8b2befd56d4040b881197bca20344d6e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.284805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] RBAC: Authorizing `identity:get_user(user_id=8b2befd56d4040b881197bca20344d6e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:54.286297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:54.288194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 128/245] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:54 2022] GET /identity/v3/users/8b2befd56d4040b881197bca20344d6e => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:54.293943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba6aae-a38a-498b-8423-9ee91550023b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:54.400272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba6aae-a38a-498b-8423-9ee91550023b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:54.435120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a611f6-3cbd-4e43-9596-12b6f3f23d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:54.436193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a611f6-3cbd-4e43-9596-12b6f3f23d19 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.436697 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a611f6-3cbd-4e43-9596-12b6f3f23d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.437396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a611f6-3cbd-4e43-9596-12b6f3f23d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.456845 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:54.515889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a611f6-3cbd-4e43-9596-12b6f3f23d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:54.517452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a611f6-3cbd-4e43-9596-12b6f3f23d19 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:54.580610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:54.629423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 118/246] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:54.630730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954aee7-00b3-4c17-84fa-59f83414fe4c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.631023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954aee7-00b3-4c17-84fa-59f83414fe4c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.631331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954aee7-00b3-4c17-84fa-59f83414fe4c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.632607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 119/247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:54.639650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fa0bb-4cd7-4c3d-92d0-fcbc34de524f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.640037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fa0bb-4cd7-4c3d-92d0-fcbc34de524f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.640495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fa0bb-4cd7-4c3d-92d0-fcbc34de524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.664443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f74fa0bb-4cd7-4c3d-92d0-fcbc34de524f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:54.665648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f74fa0bb-4cd7-4c3d-92d0-fcbc34de524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:54.676894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:54.717762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:54.720811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2ed8f-48a6-4774-87d6-da821fa1209b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:54.750714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 120/248] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:54.756138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fa0bb-4cd7-4c3d-92d0-fcbc34de524f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:54.760114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:54.760828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:54.761114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:54.761389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:54.773467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:54.779274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:54.780259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:54.781090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 129/249] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:54.788086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:54.929210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fa0bb-4cd7-4c3d-92d0-fcbc34de524f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.017283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604419cb-7883-43d4-8476-ba83d569d08f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.018691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.019372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.019652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.019923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.021791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:55.027312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:55.029641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 121/250] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:54 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:55.036141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:55.073194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:55.086435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.087066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.087726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.088075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.089052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:55.089368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:55.090955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:55.110779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 130/251] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:55.116774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:55.127547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:55.225128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c4494-14e6-4874-a243-9eeb8b362b2e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.261312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.261939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.262225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.262511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.263284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] RBAC: Adding query filter params (role.id=1953445516ad4bdf957eed35d5c68504, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=8b2befd56d4040b881197bca20344d6e) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:55.263603 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:55.265776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:55.271721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 122/252] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:55 2022] GET /identity/v3/role_assignments?user.id=8b2befd56d4040b881197bca20344d6e&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=1953445516ad4bdf957eed35d5c68504 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:55.272420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:55.345246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176a4b6-9962-4861-840e-2eddec1f7e15 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.397794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c587058-207e-4c3f-92d3-ed0c6b4286b6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.415897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:55.446497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731075a-4faf-4016-9d51-f5d6054138f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.447153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731075a-4faf-4016-9d51-f5d6054138f2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.447462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731075a-4faf-4016-9d51-f5d6054138f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.447739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731075a-4faf-4016-9d51-f5d6054138f2 None admin] PATH_INFO: `/v3/users/f89e85bacb4e422d9af96269f2347e54` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.449550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731075a-4faf-4016-9d51-f5d6054138f2 None admin] RBAC: Authorizing `identity:get_user(user_id=f89e85bacb4e422d9af96269f2347e54)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:55.451016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731075a-4faf-4016-9d51-f5d6054138f2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:55.452760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 131/253] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:55 2022] GET /identity/v3/users/f89e85bacb4e422d9af96269f2347e54 => generated 297 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:55.453911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac4d9a-9619-4e28-bfa4-432395e113ef None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.454205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac4d9a-9619-4e28-bfa4-432395e113ef None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.454485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac4d9a-9619-4e28-bfa4-432395e113ef None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.455945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 132/254] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:55.460051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac4d9a-9619-4e28-bfa4-432395e113ef None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:55.464094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faed6c-588e-4619-8c13-951fb335431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.465814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faed6c-588e-4619-8c13-951fb335431a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.466290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faed6c-588e-4619-8c13-951fb335431a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.466747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faed6c-588e-4619-8c13-951fb335431a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.481931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faed6c-588e-4619-8c13-951fb335431a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-97faed6c-588e-4619-8c13-951fb335431a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:55.488492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:55.489376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:29:55.490158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:55.492631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 123/255] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:29:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:55.494596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ed1d-4442-4c08-8f5d-459a53246f7d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.495023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ed1d-4442-4c08-8f5d-459a53246f7d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.495772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ed1d-4442-4c08-8f5d-459a53246f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.518292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9917ed1d-4442-4c08-8f5d-459a53246f7d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:55.520175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9917ed1d-4442-4c08-8f5d-459a53246f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:55.528890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:55.596060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 124/256] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:55.597564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917ed1d-4442-4c08-8f5d-459a53246f7d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:55.629018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac4d9a-9619-4e28-bfa4-432395e113ef None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.706598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917ed1d-4442-4c08-8f5d-459a53246f7d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.706808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.707619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.707997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.708436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.710386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:55.711999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:55.714142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 133/257] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:55 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:55.722502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:55.743592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:55.751920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.752961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.753436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.753913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.754955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:55.755499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:55.757589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:55.768971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 125/258] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:55.775345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:55.929545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfac280-0388-4c9d-9750-102aab5a81bc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.939563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5ad0-c8e9-49bf-8bdc-648d8a713a7a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:55.975219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:55.983459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:55.984285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:55.984570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:55.984846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] PATH_INFO: `/v3/users/5809cfedddbb40efbc2c2fa52896d930` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:55.986686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] RBAC: Authorizing `identity:get_user(user_id=5809cfedddbb40efbc2c2fa52896d930)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:55.988229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:55.989856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 126/259] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:55 2022] GET /identity/v3/users/5809cfedddbb40efbc2c2fa52896d930 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:55.997195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:56.002255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:56.002947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:56.003252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:56.003534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/f89e85bacb4e422d9af96269f2347e54/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:56.016666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] RBAC: Authorizing `identity:create_grant(project_id=8823c7322d7b4c53ad080968a7bfe121, user_id=f89e85bacb4e422d9af96269f2347e54, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:56.018327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:56.050101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 134/260] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:55 2022] PUT /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/f89e85bacb4e422d9af96269f2347e54/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:56.051279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:56.146342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8918e33-e364-407b-8b35-4cae14204d61 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:56.179834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:56.183518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:56.184142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:56.184423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:56.184698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:56.313830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:56.315486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:56.378874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1bb7b-4cec-414f-b597-b0e23f2f86cf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:56.379572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:56.451093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:56.451881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:56.452166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:56.452440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:56.475347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:56.483140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:56.484058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:29:56.484872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 135/261] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:29:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:56.493005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:56.510579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 127/262] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:56.517029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:56.595648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:56.627332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ab70b-cd9f-435a-8361-c003121fd413 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:56.659622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:56.670198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:56.671156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:56.673127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:56.673410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:56.677667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:56.679218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:56.680921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 128/263] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:56 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:56.684858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ae915-554d-4ee3-975d-31b913379d56 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:56.687415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:56.733467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:56.734092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:56.734375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:56.734803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:56.735521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:56.735831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:56.738252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:56.749662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 136/264] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:29:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:56.757939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:56.879708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ebad5-5701-4f3a-adcd-0579f92f6a2e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:56.962192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:56.963229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:56.964969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:56.965454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:56.966450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=5809cfedddbb40efbc2c2fa52896d930) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:56.967138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:56.975223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:56.988788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 129/265] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:56 2022] GET /identity/v3/role_assignments?user.id=5809cfedddbb40efbc2c2fa52896d930&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:57.001461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896ea66-a6c8-4878-b412-459dacae0062 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:57.012245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:57.049109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d5c66-1592-4033-a9f1-844889d7651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:57.050184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d5c66-1592-4033-a9f1-844889d7651f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.050689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d5c66-1592-4033-a9f1-844889d7651f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.051159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d5c66-1592-4033-a9f1-844889d7651f None admin] PATH_INFO: `/v3/users/5809cfedddbb40efbc2c2fa52896d930` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.052625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d5c66-1592-4033-a9f1-844889d7651f None admin] RBAC: Authorizing `identity:get_user(user_id=5809cfedddbb40efbc2c2fa52896d930)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:57.054303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d5c66-1592-4033-a9f1-844889d7651f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:57.059603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 137/266] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:56 2022] GET /identity/v3/users/5809cfedddbb40efbc2c2fa52896d930 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:57.063456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:57.142145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:57.200537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c06c-1c94-444d-bd0e-978568eb87b2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:57.250159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:57.250804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.251091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.251453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.293947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fdaa4-d5b3-4087-8be8-a7221a637e98 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.294267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fdaa4-d5b3-4087-8be8-a7221a637e98 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.294504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fdaa4-d5b3-4087-8be8-a7221a637e98 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.296203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 138/267] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:29:57.303593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32abd26-aac6-48d3-a95b-e67478037c1a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.303897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32abd26-aac6-48d3-a95b-e67478037c1a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.304254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32abd26-aac6-48d3-a95b-e67478037c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.329182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f32abd26-aac6-48d3-a95b-e67478037c1a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:57.330737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f32abd26-aac6-48d3-a95b-e67478037c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:57.354952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:57.356466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:57.413819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 139/268] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:57.430896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7288-c3c9-4019-8b81-aae50d90655b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.431551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7288-c3c9-4019-8b81-aae50d90655b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.432034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7288-c3c9-4019-8b81-aae50d90655b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.433703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 140/269] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:57.440140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac572-f773-44c8-9ace-cf5888f7cd19 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.440705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac572-f773-44c8-9ace-cf5888f7cd19 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.441180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac572-f773-44c8-9ace-cf5888f7cd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.459561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:57.470087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e88ac572-f773-44c8-9ace-cf5888f7cd19 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:57.471620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e88ac572-f773-44c8-9ace-cf5888f7cd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:57.536093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 130/270] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:29:57.541752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:57.549522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 141/271] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:57.692751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18de37-27c0-4ea6-88e2-e56d39daab82 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:57.734235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:57.734879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.735160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.735452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.736570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:57.738030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:57.739703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 131/272] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:57 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:57.745732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ac572-f773-44c8-9ace-cf5888f7cd19 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:57.773077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:57.801172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:57.803361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:57.915583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551efe72-ec42-4874-bebf-9790170ae806 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:57.916885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ac572-f773-44c8-9ace-cf5888f7cd19 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:57.959737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:57.960812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.961294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.961778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.961880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:57.962522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:57.962810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:57.963084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:57.963927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] RBAC: Adding query filter params (role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=5809cfedddbb40efbc2c2fa52896d930) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:57.964369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:57.965945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:57.973805 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:57.976505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 142/273] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:29:57 2022] GET /identity/v3/role_assignments?user.id=5809cfedddbb40efbc2c2fa52896d930&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:57.977396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:57.980143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:57.981053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:57.981858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:57.984776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 132/274] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:57.994641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:58.107787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:58.115158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:58.119162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c5072-04f1-4b7c-9afb-ec3abaaa68ad None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:58.130328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7b8d5-8fe8-4696-bba4-cedc2055b097 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:58.167390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597d969-02c7-45a1-8679-a025832ff562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:58.168030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597d969-02c7-45a1-8679-a025832ff562 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.168315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597d969-02c7-45a1-8679-a025832ff562 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.168599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597d969-02c7-45a1-8679-a025832ff562 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.169715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597d969-02c7-45a1-8679-a025832ff562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:58.171310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597d969-02c7-45a1-8679-a025832ff562 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:58.171541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781155b-289d-4abc-8c9a-e05428f9e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:58.172163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781155b-289d-4abc-8c9a-e05428f9e281 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.172447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781155b-289d-4abc-8c9a-e05428f9e281 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.172724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781155b-289d-4abc-8c9a-e05428f9e281 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.173228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 143/275] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:58.174454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a42e27-5001-41c8-adba-e35d8673bfb7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.174743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a42e27-5001-41c8-adba-e35d8673bfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.175021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a42e27-5001-41c8-adba-e35d8673bfb7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.176430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 144/276] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:58.178355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781155b-289d-4abc-8c9a-e05428f9e281 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:58.178685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781155b-289d-4abc-8c9a-e05428f9e281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:58.180160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a42e27-5001-41c8-adba-e35d8673bfb7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:58.180266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781155b-289d-4abc-8c9a-e05428f9e281 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:58.191323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 133/277] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:58.191917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd91d7-c87b-41cf-b367-fb6a05fc4a3d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.191917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd91d7-c87b-41cf-b367-fb6a05fc4a3d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.192100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd91d7-c87b-41cf-b367-fb6a05fc4a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.214332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e2dd91d7-c87b-41cf-b367-fb6a05fc4a3d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:58.215507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e2dd91d7-c87b-41cf-b367-fb6a05fc4a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:58.297025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 134/278] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:29:58.298886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd91d7-c87b-41cf-b367-fb6a05fc4a3d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:58.318247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a42e27-5001-41c8-adba-e35d8673bfb7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:58.354671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:58.355659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.356127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.356606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.357494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:58.359549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:58.388992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 145/279] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:29:58 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:29:58.410919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:58.439597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd91d7-c87b-41cf-b367-fb6a05fc4a3d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:58.480605 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:58.481838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.482322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.482756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] PATH_INFO: `/v3/users/04abdac9b4d0423b9fd27a07e8256cf9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.500363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] RBAC: Authorizing `identity:get_user(user_id=04abdac9b4d0423b9fd27a07e8256cf9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:58.502147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:58.504727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 135/280] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:58 2022] GET /identity/v3/users/04abdac9b4d0423b9fd27a07e8256cf9 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:58.510430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:58.540032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:58.597387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:58.635555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244bc85-b8ff-4227-bcf0-9cf016ea2acf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:58.672536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:58.673196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.673479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.673755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.674500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, user.id=04abdac9b4d0423b9fd27a07e8256cf9, scope.system=all) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:58.674818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:58.676323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:58.681607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 146/281] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 06:29:58 2022] GET /identity/v3/role_assignments?user.id=04abdac9b4d0423b9fd27a07e8256cf9&scope.system=all&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:58.726014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6de19-be6f-4b59-92dc-7fe6ec8eccb7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:58.773400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251494ea-f3cd-4ab7-a338-f77c127d425d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:58.774085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251494ea-f3cd-4ab7-a338-f77c127d425d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.774345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251494ea-f3cd-4ab7-a338-f77c127d425d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.774617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251494ea-f3cd-4ab7-a338-f77c127d425d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.777903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:58.788185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251494ea-f3cd-4ab7-a338-f77c127d425d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-251494ea-f3cd-4ab7-a338-f77c127d425d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:58.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:58.794216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:58.795364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 136/282] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:29:58.801753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:58.908334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:58.922785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984bbc2-3f59-4e19-afd1-a0270cfbbb74 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:58.962270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09432706-f758-4013-8151-7626082c7db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:58.962896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09432706-f758-4013-8151-7626082c7db4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.963182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09432706-f758-4013-8151-7626082c7db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.963482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09432706-f758-4013-8151-7626082c7db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.965669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d16f-726d-4825-8b1d-e35a811d38bb None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.966130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d16f-726d-4825-8b1d-e35a811d38bb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.966205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03d16f-726d-4825-8b1d-e35a811d38bb None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.969683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09432706-f758-4013-8151-7626082c7db4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:58.969683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09432706-f758-4013-8151-7626082c7db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:58.969899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 137/283] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:29:58.969971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09432706-f758-4013-8151-7626082c7db4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:58.975769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01856349-d579-47f4-8684-46ab85f61989 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:58.976135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01856349-d579-47f4-8684-46ab85f61989 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:58.976483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01856349-d579-47f4-8684-46ab85f61989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:58.980018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 147/284] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:58.985856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09432706-f758-4013-8151-7626082c7db4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:59.019862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-01856349-d579-47f4-8684-46ab85f61989 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:29:59.021379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-01856349-d579-47f4-8684-46ab85f61989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:29:59.150206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09432706-f758-4013-8151-7626082c7db4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:59.169126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 138/285] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:29:59.191093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:59.191826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:59.192109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:59.192381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] PATH_INFO: `/v3/users/8b2befd56d4040b881197bca20344d6e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:59.193575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] RBAC: Authorizing `identity:get_user(user_id=8b2befd56d4040b881197bca20344d6e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:59.195048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:59.196705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 148/286] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:58 2022] GET /identity/v3/users/8b2befd56d4040b881197bca20344d6e => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:29:59.203051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01856349-d579-47f4-8684-46ab85f61989 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:59.355639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01856349-d579-47f4-8684-46ab85f61989 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:59.395084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:59.405239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:59.405906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:59.406190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:59.406462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:59.407646 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:59.409095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:59.411816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 139/287] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:29:59 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:59.417692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:59.452823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:59.454916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:59.600384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d7353-cdcd-4897-a68c-a973e3208b07 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:59.631312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec13e7-c75a-4241-b6e8-274aa7f44a6a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:59.678783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:59.679382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:59.679672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:59.679954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:59.683887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:59.684571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:59.684858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:59.685137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/8b2befd56d4040b881197bca20344d6e/roles/1953445516ad4bdf957eed35d5c68504` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:59.691083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:29:59.691966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] RBAC: Authorizing `identity:create_grant(project_id=8823c7322d7b4c53ad080968a7bfe121, user_id=8b2befd56d4040b881197bca20344d6e, role_id=1953445516ad4bdf957eed35d5c68504)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:59.693462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:29:59.696897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:29:59.697768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:29:59.699448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 140/288] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:29:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:29:59.704117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:29:59.722717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 149/289] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:29:59 2022] PUT /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121/users/8b2befd56d4040b881197bca20344d6e/roles/1953445516ad4bdf957eed35d5c68504 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:29:59.794858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:29:59.857360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b485d-7edc-46af-90bb-26e72ec3dd9e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:29:59.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:29:59.906959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:29:59.907264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:29:59.907545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:29:59.908224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:29:59.908528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:29:59.909929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:29:59.918738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 141/290] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:29:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:29:59.924871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:00.016205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:00.040084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa8676-c2af-4f4f-9003-e245dd8c91ae None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:00.081768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:00.082424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:00.082709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:00.082990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] PATH_INFO: `/v3/users/f89e85bacb4e422d9af96269f2347e54` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:00.084207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] RBAC: Authorizing `identity:get_user(user_id=f89e85bacb4e422d9af96269f2347e54)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:00.085683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:00.087307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 150/291] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:29:59 2022] GET /identity/v3/users/f89e85bacb4e422d9af96269f2347e54 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:00.093367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:00.267973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab924e53-4ede-4bf0-b9c7-9c1e78442569 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:00.293148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:00.320076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:00.320762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:00.321047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:00.321320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:00.410873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:00.412420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:00.522928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 142/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:00.528267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:00.633363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b2401-dfb9-4189-ad5e-348719d2faac None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:00.668172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e16974-55cc-4fb1-b5be-ddb31e157a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:00.668870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e16974-55cc-4fb1-b5be-ddb31e157a2f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:00.669219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e16974-55cc-4fb1-b5be-ddb31e157a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:00.669501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e16974-55cc-4fb1-b5be-ddb31e157a2f None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:00.670765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e16974-55cc-4fb1-b5be-ddb31e157a2f None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:00.672501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e16974-55cc-4fb1-b5be-ddb31e157a2f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:00.674483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 151/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:00 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:00.680111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:00.702211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:00.812099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d7b46-e566-4d9c-8df3-2cbd997c37ca None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:00.865692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e1e65-16fc-4794-98e2-3242b0dd1ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:00.866320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1e65-16fc-4794-98e2-3242b0dd1ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:00.866603 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1e65-16fc-4794-98e2-3242b0dd1ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:00.866876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1e65-16fc-4794-98e2-3242b0dd1ba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:00.867644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1e65-16fc-4794-98e2-3242b0dd1ba0 None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=f89e85bacb4e422d9af96269f2347e54) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:00.867958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1e65-16fc-4794-98e2-3242b0dd1ba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:00.869393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1e65-16fc-4794-98e2-3242b0dd1ba0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:00.879636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 143/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:00 2022] GET /identity/v3/role_assignments?user.id=f89e85bacb4e422d9af96269f2347e54&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:01.060417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb79ea5-97e5-4fa1-88e0-2a3abd2b7f4a None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.060701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb79ea5-97e5-4fa1-88e0-2a3abd2b7f4a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.061311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb79ea5-97e5-4fa1-88e0-2a3abd2b7f4a None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.063679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 152/295] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:01.069745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802839-4f1d-4789-80fd-4aff749fd104 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.070314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802839-4f1d-4789-80fd-4aff749fd104 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.070813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802839-4f1d-4789-80fd-4aff749fd104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.097395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-48802839-4f1d-4789-80fd-4aff749fd104 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:01.099510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-48802839-4f1d-4789-80fd-4aff749fd104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:01.132074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44daf638-ff9a-4fd7-866c-f5de7a7c7c4a None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.132415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44daf638-ff9a-4fd7-866c-f5de7a7c7c4a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.132630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44daf638-ff9a-4fd7-866c-f5de7a7c7c4a None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.134241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 153/296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:01.139532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4123c62-e186-4b7c-9cd8-2bc3cf87f8ef None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.139798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4123c62-e186-4b7c-9cd8-2bc3cf87f8ef None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.140076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4123c62-e186-4b7c-9cd8-2bc3cf87f8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.160922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f4123c62-e186-4b7c-9cd8-2bc3cf87f8ef None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:01.160922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f4123c62-e186-4b7c-9cd8-2bc3cf87f8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:01.169837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:01.206422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 144/297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:01.207603 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556304e-d912-40e4-948d-d6faf5322a22 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.207888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556304e-d912-40e4-948d-d6faf5322a22 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.208236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556304e-d912-40e4-948d-d6faf5322a22 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.209624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 145/298] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:01.219755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66900816-0a11-464d-8f26-2b5032cf9093 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.220032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66900816-0a11-464d-8f26-2b5032cf9093 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.220312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66900816-0a11-464d-8f26-2b5032cf9093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.245477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-66900816-0a11-464d-8f26-2b5032cf9093 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:01.246885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 154/299] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:01.247303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-66900816-0a11-464d-8f26-2b5032cf9093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:01.252447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:01.323736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 146/300] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:01.511389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4123c62-e186-4b7c-9cd8-2bc3cf87f8ef None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:01.545852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66900816-0a11-464d-8f26-2b5032cf9093 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:01.625973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4123c62-e186-4b7c-9cd8-2bc3cf87f8ef None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:01.663646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:01.664337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.664562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.664834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.665161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66900816-0a11-464d-8f26-2b5032cf9093 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:01.677251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:01.683490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:01.684365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 155/301] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:01.688615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:01.693663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:01.703774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:01.704743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.705185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.705457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.706581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:01.708067 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:01.709665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 147/302] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:01.710487 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:01.723043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:01.801553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d9074-3901-425e-8f5c-da2370f984dc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:01.824000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7481bab-fdd1-4aa0-aa89-c72cdd2c26cb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:01.838947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:01.839612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.839893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.840445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.850774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:01.855560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:01.856436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:01.857240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 156/303] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:01.858574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:01.861001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:01.861698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:01.861894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:01.862165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:01.862840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:01.863145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:01.864578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:01.873110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 148/304] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:01.878248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:01.932576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:01.948555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:01.975599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927d036-da68-4691-8d15-2ebca3cbd54e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:01.983044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cd797-381f-4fad-83fc-d4a8f94a1ba0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:02.016182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e252dd-83f4-4684-980d-e69c38d6ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.017214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e252dd-83f4-4684-980d-e69c38d6ccfa None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.017441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e252dd-83f4-4684-980d-e69c38d6ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.017731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e252dd-83f4-4684-980d-e69c38d6ccfa None admin] PATH_INFO: `/v3/groups` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.018613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e252dd-83f4-4684-980d-e69c38d6ccfa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:02.020766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e252dd-83f4-4684-980d-e69c38d6ccfa None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:02.025228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.025831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.026118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.026397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.027084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:02.027421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:02.028809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:02.038378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 149/305] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:02.038772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 157/306] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Aug 24 06:30:01 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:02.040114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:02.040220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d03cea-ba21-4ec8-820d-f0fa8aa01be2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.040503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d03cea-ba21-4ec8-820d-f0fa8aa01be2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.040785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d03cea-ba21-4ec8-820d-f0fa8aa01be2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.042335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 158/307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:02.044154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d03cea-ba21-4ec8-820d-f0fa8aa01be2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:02.163627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe13194-e123-42fe-b53d-8567695206bd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:02.172142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d03cea-ba21-4ec8-820d-f0fa8aa01be2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:02.195921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:02.200575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:02.203688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45d753-45c6-4cfe-aa6f-62e89e8a288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.204346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45d753-45c6-4cfe-aa6f-62e89e8a288b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.204631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45d753-45c6-4cfe-aa6f-62e89e8a288b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.204912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45d753-45c6-4cfe-aa6f-62e89e8a288b None admin] PATH_INFO: `/v3/users/04abdac9b4d0423b9fd27a07e8256cf9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.206071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45d753-45c6-4cfe-aa6f-62e89e8a288b None admin] RBAC: Authorizing `identity:get_user(user_id=04abdac9b4d0423b9fd27a07e8256cf9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:02.207620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45d753-45c6-4cfe-aa6f-62e89e8a288b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:02.209970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 150/308] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:30:02 2022] GET /identity/v3/users/04abdac9b4d0423b9fd27a07e8256cf9 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:02.211622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044147f-39c5-4f23-a195-547481f57b32 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.211905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044147f-39c5-4f23-a195-547481f57b32 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.214259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044147f-39c5-4f23-a195-547481f57b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.222552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.223220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.224934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.224934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] PATH_INFO: `/v3/users/e736495431e947558963f53fe433486c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.235053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c044147f-39c5-4f23-a195-547481f57b32 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:02.236315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c044147f-39c5-4f23-a195-547481f57b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:02.243811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=e736495431e947558963f53fe433486c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:02.245301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:02.247151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 159/309] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:30:02 2022] GET /identity/v3/users/e736495431e947558963f53fe433486c => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:02.247775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:02.312580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 151/310] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:02.314059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044147f-39c5-4f23-a195-547481f57b32 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:02.362741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b8090-c882-4931-92dd-80ed937c1cf3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:02.400762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.401433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.401719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.402003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] PATH_INFO: `/v3/system/users/04abdac9b4d0423b9fd27a07e8256cf9/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.403571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=04abdac9b4d0423b9fd27a07e8256cf9, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:02.405530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:02.416420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 160/311] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Aug 24 06:30:02 2022] PUT /identity/v3/system/users/04abdac9b4d0423b9fd27a07e8256cf9/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:02.439912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044147f-39c5-4f23-a195-547481f57b32 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:02.475218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:02.482886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.483973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.484290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.484581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.485626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] RBAC: Adding query filter params (role.id=1953445516ad4bdf957eed35d5c68504, user.id=e736495431e947558963f53fe433486c, scope.system=all) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:02.486122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:02.487690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:02.493923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 152/312] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 06:30:02 2022] GET /identity/v3/role_assignments?user.id=e736495431e947558963f53fe433486c&scope.system=all&role.id=1953445516ad4bdf957eed35d5c68504 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:02.591952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:02.646325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:02.705719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d44a22-006a-460d-b235-e397cae923c8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:02.740763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.741417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.741702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.741977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.753124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:02.758549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:02.760584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:02.762435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 161/313] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:02.765928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:02.877602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213a671-6d39-4dbe-98f6-008c71d11b50 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:02.912866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:02.913483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:02.913765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:02.914043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:02.914779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:02.915092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:02.916563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:02.925512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 153/314] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:02.931125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:03.039854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb3581-98cc-46fb-bfd9-5f5dd5403aeb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:03.042959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:03.076538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:03.077219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:03.077497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:03.077770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] PATH_INFO: `/v3/users/8b2befd56d4040b881197bca20344d6e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:03.078914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] RBAC: Authorizing `identity:get_user(user_id=8b2befd56d4040b881197bca20344d6e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:03.080407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:03.082445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 162/315] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:30:02 2022] GET /identity/v3/users/8b2befd56d4040b881197bca20344d6e => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:03.087916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:03.154679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:03.202942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df5ac-c116-433b-ab9d-f17600c8557f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:03.241817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:03.242438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:03.242717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:03.242988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:03.312330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:03.314059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:03.369291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:03.415901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 154/316] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:03.421682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:03.531456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527f04f-96fc-4da8-abea-24dd6943c1cd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:03.570075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2295b4-75a4-4318-9358-2d8ae7c3aa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:03.570633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2295b4-75a4-4318-9358-2d8ae7c3aa7a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:03.570914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2295b4-75a4-4318-9358-2d8ae7c3aa7a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:03.571211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2295b4-75a4-4318-9358-2d8ae7c3aa7a None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:03.572408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2295b4-75a4-4318-9358-2d8ae7c3aa7a None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:03.573867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2295b4-75a4-4318-9358-2d8ae7c3aa7a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:03.575555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 163/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:03 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:03.582025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:03.691588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0584e7-d54f-4064-add6-d98417323487 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:03.727318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5771b-be24-4423-8a39-34cec7abfe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:03.727938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5771b-be24-4423-8a39-34cec7abfe70 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:03.728218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5771b-be24-4423-8a39-34cec7abfe70 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:03.728487 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5771b-be24-4423-8a39-34cec7abfe70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:03.729224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5771b-be24-4423-8a39-34cec7abfe70 None admin] RBAC: Adding query filter params (role.id=1953445516ad4bdf957eed35d5c68504, scope.project.id=8823c7322d7b4c53ad080968a7bfe121, user.id=8b2befd56d4040b881197bca20344d6e) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:03.729531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5771b-be24-4423-8a39-34cec7abfe70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:03.730960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5771b-be24-4423-8a39-34cec7abfe70 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:03.737853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 155/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:03 2022] GET /identity/v3/role_assignments?user.id=8b2befd56d4040b881197bca20344d6e&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=1953445516ad4bdf957eed35d5c68504 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:04.078273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16b648-daec-4ac9-9e46-d79fb2b892fb None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.078826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16b648-daec-4ac9-9e46-d79fb2b892fb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.078826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16b648-daec-4ac9-9e46-d79fb2b892fb None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.080516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 164/319] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:04.085916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a711d57-104e-496a-b225-5509dd12d384 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.086215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a711d57-104e-496a-b225-5509dd12d384 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.086492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a711d57-104e-496a-b225-5509dd12d384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.105245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5a711d57-104e-496a-b225-5509dd12d384 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:04.106438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5a711d57-104e-496a-b225-5509dd12d384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:04.122061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:04.178966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 156/320] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:04.393243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807fa4f-8c3a-4afd-8d67-1f1eadf14b96 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.393427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807fa4f-8c3a-4afd-8d67-1f1eadf14b96 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.393708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807fa4f-8c3a-4afd-8d67-1f1eadf14b96 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.395254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 165/321] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:04.400459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba268a4-3197-4273-8331-a025d4a0b35a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.400918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba268a4-3197-4273-8331-a025d4a0b35a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.400990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba268a4-3197-4273-8331-a025d4a0b35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.419862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-eba268a4-3197-4273-8331-a025d4a0b35a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:04.421194 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eba268a4-3197-4273-8331-a025d4a0b35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:04.454762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807fa4f-8c3a-4afd-8d67-1f1eadf14b96 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:04.493370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 157/322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:04.494724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd771ea-857d-42ef-ab43-af95695870e9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.495016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd771ea-857d-42ef-ab43-af95695870e9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.495324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd771ea-857d-42ef-ab43-af95695870e9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.496690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 158/323] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:04.504556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb80f8-a444-4d17-991a-1a8bf4cae694 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.504842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb80f8-a444-4d17-991a-1a8bf4cae694 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.505123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb80f8-a444-4d17-991a-1a8bf4cae694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.508080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:04.523684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7deb80f8-a444-4d17-991a-1a8bf4cae694 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:04.524873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7deb80f8-a444-4d17-991a-1a8bf4cae694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:04.574733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807fa4f-8c3a-4afd-8d67-1f1eadf14b96 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:04.596127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 159/324] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:04.612986 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:04.613622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.613899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.614169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.615283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:04.616793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:04.618481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 166/325] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:04.624087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb80f8-a444-4d17-991a-1a8bf4cae694 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:04.636325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:04.726930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb80f8-a444-4d17-991a-1a8bf4cae694 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:04.761271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:04.762348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:04.763112 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.763426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.763702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] PATH_INFO: `/v3/groups` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.764484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:04.765970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:04.786175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 160/326] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Aug 24 06:30:04 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:04.864883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:04.892057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06bc35-7987-48d6-9a94-6eabc289cf8d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:04.902984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:04.933198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:04.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:04.934125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:04.934403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:04.946123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:04.950431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:04.956649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:04.957638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:04.958567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 167/327] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:04.965046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:04.992012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da28a-a2c3-42c8-8947-ace3469273f3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:05.039230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:05.039897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:05.040177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:05.040453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:05.052791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:05.058898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:05.059878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:05.060692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 161/328] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:05.069745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:05.097860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f3091-061e-47f1-8a83-4415d3133f2b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:05.138006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:05.138635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:05.138920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:05.139414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:05.140018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:05.140397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:05.141815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:05.153681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 168/329] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:05.162171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:05.199930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec1685-bead-40da-8784-eac83f15673c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:05.200462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:05.239216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:05.252808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:05.253546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:05.254391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:05.254693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:05.257957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:05.257957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:05.257957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:05.270078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 162/330] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:05.276343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:05.296843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37375d54-c250-4107-a352-31725bc2d6c7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:05.338853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:05.339805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:05.340093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:05.340370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] PATH_INFO: `/v3/users/04abdac9b4d0423b9fd27a07e8256cf9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:05.342282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] RBAC: Authorizing `identity:get_user(user_id=04abdac9b4d0423b9fd27a07e8256cf9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:05.344540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:05.347212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 169/331] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:30:05 2022] GET /identity/v3/users/04abdac9b4d0423b9fd27a07e8256cf9 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:05.353992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:05.398747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb040ce9-5ab2-4e2e-adc3-4308f879b34b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:05.440637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:05.441315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:05.441550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:05.441828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] PATH_INFO: `/v3/users/e736495431e947558963f53fe433486c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:05.442986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] RBAC: Authorizing `identity:get_user(user_id=e736495431e947558963f53fe433486c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:05.444499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:05.446467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 163/332] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:30:05 2022] GET /identity/v3/users/e736495431e947558963f53fe433486c => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:05.452317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:05.492765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb041f-b147-435f-8cc3-58dc22aa369a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:05.507682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:05.532556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:05.542038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14dcb-4f28-46f5-9260-4b020519754f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:05.542810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14dcb-4f28-46f5-9260-4b020519754f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:05.543215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14dcb-4f28-46f5-9260-4b020519754f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:05.543591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14dcb-4f28-46f5-9260-4b020519754f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:05.544686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14dcb-4f28-46f5-9260-4b020519754f None admin] RBAC: Adding query filter params (role.id=ffb20dd7d8254b1b960acfe117faa5c3, user.id=04abdac9b4d0423b9fd27a07e8256cf9, scope.system=all) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:05.545092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14dcb-4f28-46f5-9260-4b020519754f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:05.547515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14dcb-4f28-46f5-9260-4b020519754f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:05.566919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 170/333] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 06:30:05 2022] GET /identity/v3/role_assignments?user.id=04abdac9b4d0423b9fd27a07e8256cf9&scope.system=all&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:05.594124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19485ce3-35cf-4abe-bd35-0689c5ce1310 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:05.641198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fd0e9-79bb-4a3b-aa8b-cbae214514d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:05.642269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd0e9-79bb-4a3b-aa8b-cbae214514d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:05.642746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd0e9-79bb-4a3b-aa8b-cbae214514d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:05.643238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd0e9-79bb-4a3b-aa8b-cbae214514d0 None admin] PATH_INFO: `/v3/system/users/e736495431e947558963f53fe433486c/roles/1953445516ad4bdf957eed35d5c68504` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:05.645007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fd0e9-79bb-4a3b-aa8b-cbae214514d0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e736495431e947558963f53fe433486c, role_id=1953445516ad4bdf957eed35d5c68504)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:05.646653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fd0e9-79bb-4a3b-aa8b-cbae214514d0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:05.657579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 164/334] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Aug 24 06:30:05 2022] PUT /identity/v3/system/users/e736495431e947558963f53fe433486c/roles/1953445516ad4bdf957eed35d5c68504 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:06.995954 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c013f11-b053-4a67-9948-561b894b1203 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:06.996512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c013f11-b053-4a67-9948-561b894b1203 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:06.996512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c013f11-b053-4a67-9948-561b894b1203 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:06.997970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 171/335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:07.004063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28fc75-7a5f-47aa-8e0a-8d3e8fabeded None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:07.004534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28fc75-7a5f-47aa-8e0a-8d3e8fabeded None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:07.004618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28fc75-7a5f-47aa-8e0a-8d3e8fabeded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:07.032289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ed28fc75-7a5f-47aa-8e0a-8d3e8fabeded None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:07.033534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ed28fc75-7a5f-47aa-8e0a-8d3e8fabeded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:07.114121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 165/336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:07.276182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d55ac3-ab48-4661-b6f6-4c7ce995d776 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:07.276372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d55ac3-ab48-4661-b6f6-4c7ce995d776 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:07.276653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d55ac3-ab48-4661-b6f6-4c7ce995d776 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:07.278291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 172/337] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:07.295775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93c217-5f76-4a0e-b096-9a5361fff00d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:07.296122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93c217-5f76-4a0e-b096-9a5361fff00d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:07.296407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93c217-5f76-4a0e-b096-9a5361fff00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:07.321340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ab93c217-5f76-4a0e-b096-9a5361fff00d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:07.322595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ab93c217-5f76-4a0e-b096-9a5361fff00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:07.399028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:07.420367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 166/338] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:07.673082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d55ac3-ab48-4661-b6f6-4c7ce995d776 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:07.790156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d55ac3-ab48-4661-b6f6-4c7ce995d776 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:07.831090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:07.832142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:07.832625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:07.833122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:07.848467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:07.854079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:07.854972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:07.855811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:07.859960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 173/339] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:07.860324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93c217-5f76-4a0e-b096-9a5361fff00d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:07.866324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:07.896011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:08.075812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93c217-5f76-4a0e-b096-9a5361fff00d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:08.089764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0577-8ca1-4b72-9fe5-e4898c080246 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:08.158891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4eafaa-d9ae-4e1c-bb00-577a69f15480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:08.159582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eafaa-d9ae-4e1c-bb00-577a69f15480 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.159861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eafaa-d9ae-4e1c-bb00-577a69f15480 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.160143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eafaa-d9ae-4e1c-bb00-577a69f15480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.168505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839740bd-654a-4327-a61a-faf08a173c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:08.169145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839740bd-654a-4327-a61a-faf08a173c2f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.169497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839740bd-654a-4327-a61a-faf08a173c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.169782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839740bd-654a-4327-a61a-faf08a173c2f None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.170478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839740bd-654a-4327-a61a-faf08a173c2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:08.170852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839740bd-654a-4327-a61a-faf08a173c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:08.172291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839740bd-654a-4327-a61a-faf08a173c2f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:08.174219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4eafaa-d9ae-4e1c-bb00-577a69f15480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4d4eafaa-d9ae-4e1c-bb00-577a69f15480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:08.179045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:08.179976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:08.181191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 167/340] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:30:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:08.182054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc117bc-6aa7-4c67-8ebd-aa1d6eba4ed0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.182305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc117bc-6aa7-4c67-8ebd-aa1d6eba4ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.182580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc117bc-6aa7-4c67-8ebd-aa1d6eba4ed0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.188743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 168/341] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:08.191071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc117bc-6aa7-4c67-8ebd-aa1d6eba4ed0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:08.191801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 174/342] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:08.194016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add759-8dfd-4433-a18e-4b3b8624b499 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.194307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add759-8dfd-4433-a18e-4b3b8624b499 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.194590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2add759-8dfd-4433-a18e-4b3b8624b499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.226834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f2add759-8dfd-4433-a18e-4b3b8624b499 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:08.228102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f2add759-8dfd-4433-a18e-4b3b8624b499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:08.271245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:08.302616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 175/343] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:08.303613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2add759-8dfd-4433-a18e-4b3b8624b499 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:08.314706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:08.365501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc117bc-6aa7-4c67-8ebd-aa1d6eba4ed0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:08.410248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2add759-8dfd-4433-a18e-4b3b8624b499 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:08.433595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:08.434655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.435130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.435665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.437122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:08.437626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:08.439648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:08.447121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f700c-111c-45b4-85cc-3c27511e889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:08.447779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f700c-111c-45b4-85cc-3c27511e889f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.448064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f700c-111c-45b4-85cc-3c27511e889f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.448344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f700c-111c-45b4-85cc-3c27511e889f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.458186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 169/344] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:08.464171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:08.525607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f700c-111c-45b4-85cc-3c27511e889f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:08.527135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f700c-111c-45b4-85cc-3c27511e889f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:08.537489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:08.654623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6606d-9c65-4ef1-933f-36065445807d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:08.669450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:08.713989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bd039-a2be-40e0-a687-3448e80592e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:08.713989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd039-a2be-40e0-a687-3448e80592e2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.713989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd039-a2be-40e0-a687-3448e80592e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.713989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd039-a2be-40e0-a687-3448e80592e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.714915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 176/345] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:08.716638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcefdf-e2b0-4279-9faa-35cd485972ee None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.717224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcefdf-e2b0-4279-9faa-35cd485972ee None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.717752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcefdf-e2b0-4279-9faa-35cd485972ee None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.719655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 177/346] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:08.720567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcefdf-e2b0-4279-9faa-35cd485972ee None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:08.809756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bd039-a2be-40e0-a687-3448e80592e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:08.811621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bd039-a2be-40e0-a687-3448e80592e2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:08.857476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcefdf-e2b0-4279-9faa-35cd485972ee None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:08.868790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:08.900612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:08.902424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.902910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.903456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.917948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:08.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:08.923824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:08.924656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:08.926153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 178/347] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:08.926965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:08.946398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 170/348] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:30:08.948362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec1c67-d362-421c-bea3-0fbdfb31b9ba None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:08.954371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec1c67-d362-421c-bea3-0fbdfb31b9ba None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:08.954371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec1c67-d362-421c-bea3-0fbdfb31b9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:08.967054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:08.977053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6bec1c67-d362-421c-bea3-0fbdfb31b9ba None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:08.978242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6bec1c67-d362-421c-bea3-0fbdfb31b9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:09.054213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 171/349] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:09.055234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec1c67-d362-421c-bea3-0fbdfb31b9ba None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.064247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae27b14-f747-42f0-88bf-d63e4956564f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.101830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.103357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.103839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.105563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.107038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.109140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.115498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 179/350] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:08 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:09.116679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.155965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:09.162026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec1c67-d362-421c-bea3-0fbdfb31b9ba None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.199249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.199858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.200144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.200421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.201105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:09.201413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.202865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.205607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:09.223582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 172/351] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:09.224427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.235560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab737e7e-4ebe-4a07-b1e8-6f197fc8b929 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.274220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.275255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.275759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.276314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.277725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.279431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.282125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 180/352] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:09.283273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.332350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b63dae-da59-4d3b-9ed0-7c26ea9be148 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.368573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.369218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.369501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.369778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.377083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.378618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.380324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 173/353] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:09.381294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.406261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435629-b860-458e-aaba-10478e3acfcf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.409384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:09.446700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:09.449349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379fdc2-1562-43c6-9e26-50fdfc54e352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.450313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379fdc2-1562-43c6-9e26-50fdfc54e352 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.450896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379fdc2-1562-43c6-9e26-50fdfc54e352 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.451446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379fdc2-1562-43c6-9e26-50fdfc54e352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.489083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abc6bd-81c9-4b1d-9cd0-acd595316d96 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.525427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.526198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.526485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.526861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.528037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.529498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.532531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 174/354] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:09.533350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.541276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379fdc2-1562-43c6-9e26-50fdfc54e352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.542787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379fdc2-1562-43c6-9e26-50fdfc54e352 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.647642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243668d5-6735-484d-8869-959e39188ac5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.657291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 181/355] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:09.659752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ebdf1-4d81-421b-9e53-553f92bf95a7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.660001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ebdf1-4d81-421b-9e53-553f92bf95a7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.660287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ebdf1-4d81-421b-9e53-553f92bf95a7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.662730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 182/356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:09.662971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326ebc-327d-44bd-b7d8-d98da8f9ec72 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.663292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326ebc-327d-44bd-b7d8-d98da8f9ec72 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.663573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326ebc-327d-44bd-b7d8-d98da8f9ec72 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.664908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 183/357] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:09.668049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b326ebc-327d-44bd-b7d8-d98da8f9ec72 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.672004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:09.690425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:09.697383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.698314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.698779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.699225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.700230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:09.700715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.702330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.709618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 175/358] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:09.710430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.815307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b326ebc-327d-44bd-b7d8-d98da8f9ec72 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.856403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a895d78-42ee-4e71-b496-9be6baa11f8a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:09.856785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566679eb-dd65-4780-bff1-83ca0afe6c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.857450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566679eb-dd65-4780-bff1-83ca0afe6c7c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.857734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566679eb-dd65-4780-bff1-83ca0afe6c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.858057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566679eb-dd65-4780-bff1-83ca0afe6c7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.872637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566679eb-dd65-4780-bff1-83ca0afe6c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-566679eb-dd65-4780-bff1-83ca0afe6c7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:09.877796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:09.878725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:30:09.879687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:09.880310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 184/359] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:09.881454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca9c83-3eb7-4b94-8abd-6c9357ce97e9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.881747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca9c83-3eb7-4b94-8abd-6c9357ce97e9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.882030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca9c83-3eb7-4b94-8abd-6c9357ce97e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.902856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:09.903516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:09.903801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:09.904077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:09.904825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:09.904995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-97ca9c83-3eb7-4b94-8abd-6c9357ce97e9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:09.905219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:09.906555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-97ca9c83-3eb7-4b94-8abd-6c9357ce97e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:09.906761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:09.913911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 176/360] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:09.915006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:09.995581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:10.011271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 185/361] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:10.011271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b830fb-d984-4a16-916e-a2200fd7d0ab None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.011271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b830fb-d984-4a16-916e-a2200fd7d0ab None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.011271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b830fb-d984-4a16-916e-a2200fd7d0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.015872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:10.036583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-56b830fb-d984-4a16-916e-a2200fd7d0ab None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:10.041016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-56b830fb-d984-4a16-916e-a2200fd7d0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:10.113469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5ef93-d19b-439c-b640-0eda66f85628 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.134352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 186/362] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:10.135110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b830fb-d984-4a16-916e-a2200fd7d0ab None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.178880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.180264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.180767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.181233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.182565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:10.185955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:10.195577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 177/363] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:09 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:10.213026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.291319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b830fb-d984-4a16-916e-a2200fd7d0ab None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.323620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:10.347936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.348577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.348862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.349139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.349834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:10.350140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:10.351573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:10.360531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 187/364] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:10.366097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.394017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c5cc1-a454-4846-8ef0-cc78e9afc504 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.427634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:10.448832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.449876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.450378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.450848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.452373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:10.454028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:10.456249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 178/365] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:10.457285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.506437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a5e0-3684-48d5-abb9-23abd8145215 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.545060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.545605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.545936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.546210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] PATH_INFO: `/v3/users/e736495431e947558963f53fe433486c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.547438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] RBAC: Authorizing `identity:get_user(user_id=e736495431e947558963f53fe433486c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:10.548897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:10.550813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 188/366] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/users/e736495431e947558963f53fe433486c => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:10.551641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.619341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:10.657629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f526f-f5fb-4781-8225-acf2fee3a582 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.669503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a372e-ecc0-4fc2-b362-994570b11a27 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.697331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.697949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.698234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.698511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.699278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=8823c7322d7b4c53ad080968a7bfe121) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:10.699646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:10.701175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:10.708420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.708775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 189/367] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:10.709060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.709337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.709589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.709685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.721562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:10.726899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:10.727905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:10.728720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 179/368] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:10.729384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.803691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:10.850837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce9c94-957a-4677-8803-0c5e2ed5de04 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.889008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68566209-8220-451c-8549-2935c03af69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.889727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566209-8220-451c-8549-2935c03af69c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.890004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566209-8220-451c-8549-2935c03af69c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.890339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566209-8220-451c-8549-2935c03af69c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.902113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68566209-8220-451c-8549-2935c03af69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-68566209-8220-451c-8549-2935c03af69c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:10.907925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:10.908874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:10.910104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 180/369] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:10.910781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68566209-8220-451c-8549-2935c03af69c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:10.912928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dbe60-206e-4ec3-bc94-e42185892702 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:10.952984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:10.989125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:10.990637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:10.991415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:10.992215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:10.993855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] RBAC: Adding query filter params (role.id=1953445516ad4bdf957eed35d5c68504, user.id=e736495431e947558963f53fe433486c, scope.system=all) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:10.994679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:10.997934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:11.019632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 190/370] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/role_assignments?user.id=e736495431e947558963f53fe433486c&scope.system=all&role.id=1953445516ad4bdf957eed35d5c68504 => generated 510 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:11.021053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:11.040890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68566209-8220-451c-8549-2935c03af69c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:11.057517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:11.083848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:11.084480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:11.084759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:11.085029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:11.085867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:11.086179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:11.087638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:11.097817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 181/371] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:11.104537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:11.220016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e5859-01ac-4be6-b1a7-ea6fc0ca37f0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:11.300805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07705eda-2926-40e0-a685-820edb67e924 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:11.316085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:11.317092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:11.317670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:11.318141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:11.370892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:11.371547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:11.371833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:11.372115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:11.372801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:11.373108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:11.374652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:11.387766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 191/372] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:11.393795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:11.422649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:11.444548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:11.471031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:11.472545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:11.542177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cb2d-158e-47b8-8a82-bcd023317d23 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:11.596729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:11.597277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:11.597562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:11.597838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:11.705548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 182/373] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:30:11.707826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:11.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:11.710574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:11.776496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:11.817160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:11.850162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 192/374] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:30:11.852644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b201a-6e15-47a8-acaa-ec48c869a792 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:11.856146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:11.894169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:11.894825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:11.895108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:11.895412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:11.896673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:11.898177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:11.900255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 183/375] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:11 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:11.908437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:12.004979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b544641-6b42-42f7-9918-372b7d7cba75 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:12.045099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69f83b-938f-4b9d-b9f9-6be5c12b6808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:12.045848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69f83b-938f-4b9d-b9f9-6be5c12b6808 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.046131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69f83b-938f-4b9d-b9f9-6be5c12b6808 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.046402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69f83b-938f-4b9d-b9f9-6be5c12b6808 None admin] PATH_INFO: `/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.048109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69f83b-938f-4b9d-b9f9-6be5c12b6808 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0ff1eb049e4a50aede95b2f05e94b7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:12.049750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69f83b-938f-4b9d-b9f9-6be5c12b6808 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:12.051952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 193/376] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:11 2022] GET /identity/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:12.057751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eca60-5a35-4a63-a07b-b41cead95e17 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.058035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eca60-5a35-4a63-a07b-b41cead95e17 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.058306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649eca60-5a35-4a63-a07b-b41cead95e17 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.059906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 194/377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:12.060685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649eca60-5a35-4a63-a07b-b41cead95e17 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:12.075926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:12.107940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e5dc0-6b1d-493c-9fb2-0737212f86ee None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:12.148134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe98d7e-9562-4a85-84c5-777cadca7603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:12.148775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe98d7e-9562-4a85-84c5-777cadca7603 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.149052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe98d7e-9562-4a85-84c5-777cadca7603 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.149321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe98d7e-9562-4a85-84c5-777cadca7603 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.150427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe98d7e-9562-4a85-84c5-777cadca7603 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:12.152949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe98d7e-9562-4a85-84c5-777cadca7603 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:12.154954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 184/378] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:11 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:12.156341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890fb7d-aea7-4ef7-983d-ec5f9ff9f2f9 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.156652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890fb7d-aea7-4ef7-983d-ec5f9ff9f2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.156927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890fb7d-aea7-4ef7-983d-ec5f9ff9f2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.177135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f890fb7d-aea7-4ef7-983d-ec5f9ff9f2f9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:12.178368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f890fb7d-aea7-4ef7-983d-ec5f9ff9f2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:12.196051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649eca60-5a35-4a63-a07b-b41cead95e17 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:12.196243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:12.240108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:12.241162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.241544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.242022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] PATH_INFO: `/v3/groups/238633eff27244c58d5dcc731b172500` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.248692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] RBAC: Authorizing `identity:get_group(group_id=238633eff27244c58d5dcc731b172500)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:12.250949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:12.252883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 195/379] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:12 2022] GET /identity/v3/groups/238633eff27244c58d5dcc731b172500 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:12.253854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:12.276359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 185/380] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:12.278613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890fb7d-aea7-4ef7-983d-ec5f9ff9f2f9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:12.346848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:12.389479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71716741-10f7-4dd9-babb-d78af6a2d961 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:12.419270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890fb7d-aea7-4ef7-983d-ec5f9ff9f2f9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:12.432230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:12.433005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.433408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.433780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.434874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=8823c7322d7b4c53ad080968a7bfe121) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:12.434988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:12.436639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:12.443730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 196/381] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:12 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:12.470599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344972c-54b9-4e90-9806-4721c5506a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:12.471566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344972c-54b9-4e90-9806-4721c5506a51 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.471989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344972c-54b9-4e90-9806-4721c5506a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.472399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344972c-54b9-4e90-9806-4721c5506a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.473315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344972c-54b9-4e90-9806-4721c5506a51 None admin] RBAC: Adding query filter params (group.id=238633eff27244c58d5dcc731b172500, role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=4d0ff1eb049e4a50aede95b2f05e94b7) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:12.473814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344972c-54b9-4e90-9806-4721c5506a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:12.475494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344972c-54b9-4e90-9806-4721c5506a51 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:12.483025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 186/382] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:12 2022] GET /identity/v3/role_assignments?group.id=238633eff27244c58d5dcc731b172500&scope.project.id=4d0ff1eb049e4a50aede95b2f05e94b7&role.id=c64ac97b1795434390dc18ab162aec39 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:12.744580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:12.823445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2b55-27f8-45f5-9914-8a0059ae83dc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.824084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2b55-27f8-45f5-9914-8a0059ae83dc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.824572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2b55-27f8-45f5-9914-8a0059ae83dc None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.831521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 187/383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:12.833495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276db9d9-9dac-4e9a-9d99-6c27ddd883df None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.833978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276db9d9-9dac-4e9a-9d99-6c27ddd883df None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.835613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276db9d9-9dac-4e9a-9d99-6c27ddd883df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.853971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:12.866251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-276db9d9-9dac-4e9a-9d99-6c27ddd883df None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:12.872427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-276db9d9-9dac-4e9a-9d99-6c27ddd883df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:12.876141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86be4a-1355-4b00-b6b9-c5b2b4cb0046 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:12.917237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:12.918741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:12.920632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:12.921157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:12.925974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:12.950153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:12.962282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:12.963260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:12.964069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:12.968436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 197/384] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:12.974329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:13.017342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 188/385] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:13.117793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9573d7-c4da-4b1f-a699-feef2de89268 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:13.159437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:13.160424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:13.160897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:13.161383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:13.162257 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:13.162767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:13.164411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:13.174140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 198/386] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:13.180145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276db9d9-9dac-4e9a-9d99-6c27ddd883df None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:13.300583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276db9d9-9dac-4e9a-9d99-6c27ddd883df None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:13.346516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:13.347158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:13.347466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:13.347741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:13.351389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:13.352894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:13.354805 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 189/387] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:13 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:13.360603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:13.426760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:13.475845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39aab7-97bf-453f-a955-12353f1382fb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:13.514733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:13.516189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:13.516480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:13.516756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:13.518724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:13.520241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:13.522260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 199/388] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:13 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:13.528147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:13.532307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:13.588667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:13.667475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1340e2-623a-4b93-972f-67834ed3509f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:13.711495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f5f7a-e29a-46a1-9742-8432bbb70546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:13.713026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5f7a-e29a-46a1-9742-8432bbb70546 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:13.713526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5f7a-e29a-46a1-9742-8432bbb70546 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:13.714012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f5f7a-e29a-46a1-9742-8432bbb70546 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/groups/2e8d6a65ade747789d78a70815caa921/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:13.716188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f5f7a-e29a-46a1-9742-8432bbb70546 None admin] RBAC: Authorizing `identity:create_grant(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9, group_id=2e8d6a65ade747789d78a70815caa921, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:13.718289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f5f7a-e29a-46a1-9742-8432bbb70546 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:13.727456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4a02-0a0b-467b-8d73-445aa0d5b25e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:13.739397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 190/389] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 06:30:13 2022] PUT /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/groups/2e8d6a65ade747789d78a70815caa921/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:13.741433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f38d7-2230-4ae8-abcd-551b989c31bc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:13.741923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f38d7-2230-4ae8-abcd-551b989c31bc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:13.742392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f38d7-2230-4ae8-abcd-551b989c31bc None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:13.747210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 191/390] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:13.753418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dcf3c-ad70-4f19-8188-76e1602200cd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:13.754051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dcf3c-ad70-4f19-8188-76e1602200cd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:13.754576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dcf3c-ad70-4f19-8188-76e1602200cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:13.761712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:13.777815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0d3dcf3c-ad70-4f19-8188-76e1602200cd None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:13.779552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0d3dcf3c-ad70-4f19-8188-76e1602200cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:13.784656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:13.785333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:13.785623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:13.785903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:13.808384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:13.820678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:13.821938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:13.822872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 200/391] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:30:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:13.828513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:13.862141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:13.870959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 192/392] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:14.009690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac704f1c-a0b2-40f4-b63e-00159468ea74 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:14.053151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:14.053771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.054053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.054324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.055003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:14.055358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:14.056763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:14.068263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 201/393] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:14.073075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dcf3c-ad70-4f19-8188-76e1602200cd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:14.191806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dcf3c-ad70-4f19-8188-76e1602200cd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:14.236217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:14.236875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.237295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.237574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.238707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:14.240254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:14.244522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 193/394] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:14 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:14.248381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:14.315406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:14.353540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:14.389877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32651a-bae6-44e0-969f-32e2550f1270 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:14.437227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:14.453326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:14.453649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.453828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.454092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.455352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:14.456803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:14.463250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 202/395] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:14 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:14.465318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:14.495257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9cc4d-c021-4ab3-89f9-7b75a378bb22 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:14.571978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:14.572639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.572922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.573192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.598672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:14.603599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:14.604502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:14.607647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 194/396] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:14.610983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:14.630333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd615d-2088-4b3e-a404-02327df9b696 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:14.646927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:14.673782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c203d33-ece9-481d-a1f9-aabf0d1162e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:14.674844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203d33-ece9-481d-a1f9-aabf0d1162e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.675365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203d33-ece9-481d-a1f9-aabf0d1162e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.675861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c203d33-ece9-481d-a1f9-aabf0d1162e8 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/groups/2e8d6a65ade747789d78a70815caa921/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.678316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c203d33-ece9-481d-a1f9-aabf0d1162e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9, group_id=2e8d6a65ade747789d78a70815caa921, role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:14.680135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c203d33-ece9-481d-a1f9-aabf0d1162e8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:14.708237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 203/397] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 06:30:14 2022] PUT /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9/groups/2e8d6a65ade747789d78a70815caa921/roles/c37a4d448aa147ebb487e7de2099e1db => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:30:14.755738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3ad0d-1fb2-43f5-856f-64675ec206c2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.756396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3ad0d-1fb2-43f5-856f-64675ec206c2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.756870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3ad0d-1fb2-43f5-856f-64675ec206c2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.759140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 204/398] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:14.765179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24442589-d004-4826-b329-693228fc77d4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.765823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24442589-d004-4826-b329-693228fc77d4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.766481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24442589-d004-4826-b329-693228fc77d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.792711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-24442589-d004-4826-b329-693228fc77d4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:14.795345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-24442589-d004-4826-b329-693228fc77d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:14.853523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0abbd-fdb2-457f-88de-ab40a72c7d48 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:14.886652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 205/399] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:14.914475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:14.933823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:14.934450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:14.934729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:14.935002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:14.935704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:14.936013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:14.937610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:14.953563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 195/400] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:14.959286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24442589-d004-4826-b329-693228fc77d4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.072161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24442589-d004-4826-b329-693228fc77d4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:15.092041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:15.111451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555558-06af-4304-897d-82b098cb2f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:15.112264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555558-06af-4304-897d-82b098cb2f67 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.112546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555558-06af-4304-897d-82b098cb2f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.112816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555558-06af-4304-897d-82b098cb2f67 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.114060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555558-06af-4304-897d-82b098cb2f67 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:15.115729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555558-06af-4304-897d-82b098cb2f67 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:15.117698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 206/401] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:14 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:15.122898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.128537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c438128-c52c-4905-9a95-a935fe109261 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.128821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c438128-c52c-4905-9a95-a935fe109261 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.129098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c438128-c52c-4905-9a95-a935fe109261 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.130499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 207/402] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:30:15.140687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c87832-8b18-4948-9f61-0ae2a3773bb1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.140961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c87832-8b18-4948-9f61-0ae2a3773bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.141240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c87832-8b18-4948-9f61-0ae2a3773bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.173666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a1c87832-8b18-4948-9f61-0ae2a3773bb1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:15.174877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a1c87832-8b18-4948-9f61-0ae2a3773bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:15.283303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 208/403] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:15.284332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c87832-8b18-4948-9f61-0ae2a3773bb1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.306848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31737b-a76d-49f0-a1d2-6235cfe80253 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:15.357793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:15.358617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.359163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.361373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.362108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:15.366046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:15.369986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 196/404] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:15 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:15.376240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.439808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c87832-8b18-4948-9f61-0ae2a3773bb1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:15.441553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:15.473937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:15.504502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:15.505145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.505421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.505692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.526942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:15.532628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:15.533782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:15.535042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 209/405] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:15.540145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.564481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984e010-dd88-4526-9a84-2687d344640f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:15.617235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:15.618190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.618626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.619065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121/groups/2e8d6a65ade747789d78a70815caa921/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.621712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] RBAC: Authorizing `identity:create_grant(project_id=8823c7322d7b4c53ad080968a7bfe121, group_id=2e8d6a65ade747789d78a70815caa921, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:15.623965 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:15.645180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 197/406] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 06:30:15 2022] PUT /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121/groups/2e8d6a65ade747789d78a70815caa921/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:15.646562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.651305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d62c-37ed-4cf1-9e59-4bcf47589b3e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:15.749151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:15.750105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.750637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.751121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.752195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:15.752666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:15.754989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:15.767174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 210/407] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:15.778830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.814299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:15.858197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f30b31-ded2-4ce6-897d-4a4bd4d923cd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:15.872329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:15.897392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:15.898088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.898373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.898647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.909301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:15.914050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a36c06-ee93-4e40-aac0-9481e77c7616 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:15.916238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:15.917320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:15.918378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 198/408] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:30:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:15.923298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:15.958392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:15.959285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:15.959899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:15.960351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] PATH_INFO: `/v3/groups/238633eff27244c58d5dcc731b172500` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:15.961392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] RBAC: Authorizing `identity:get_group(group_id=238633eff27244c58d5dcc731b172500)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:15.963016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:15.965129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 211/409] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:15 2022] GET /identity/v3/groups/238633eff27244c58d5dcc731b172500 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:15.971832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:16.048739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbc3d8-ebc9-43d0-bc19-55d15b882419 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:16.094898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:16.095819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.096255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.096678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.097518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:16.097976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:16.099547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:16.112060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 199/410] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:16.118018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:16.120398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:16.132410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c4081-3188-43a2-a5df-2fa581a9d76a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:16.169938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:16.171357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.172090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.172868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] PATH_INFO: `/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.174332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0ff1eb049e4a50aede95b2f05e94b7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:16.176013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:16.178380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 212/411] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:15 2022] GET /identity/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:16.184188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:16.205788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:16.301028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0998448-258d-4ea3-bbbe-bb40faad40d7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:16.301747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22835-bd50-48fb-90ab-3fa345aafb95 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:16.336134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03852c-a94b-4b3b-876e-1ca5d40cbd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:16.336793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03852c-a94b-4b3b-876e-1ca5d40cbd7d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.337071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03852c-a94b-4b3b-876e-1ca5d40cbd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.337363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03852c-a94b-4b3b-876e-1ca5d40cbd7d None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.338458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03852c-a94b-4b3b-876e-1ca5d40cbd7d None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:16.340052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03852c-a94b-4b3b-876e-1ca5d40cbd7d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:16.341471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fd08-7cd1-4e6c-a021-8b2e9efc1350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:16.342074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 200/412] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:16 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:16.342645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fd08-7cd1-4e6c-a021-8b2e9efc1350 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.343273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fd08-7cd1-4e6c-a021-8b2e9efc1350 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.343390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195232-ded9-4443-8df9-6c56956283be None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.343627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195232-ded9-4443-8df9-6c56956283be None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.343906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195232-ded9-4443-8df9-6c56956283be None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.344253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fd08-7cd1-4e6c-a021-8b2e9efc1350 None admin] PATH_INFO: `/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7/groups/238633eff27244c58d5dcc731b172500/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.345278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 201/413] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:16.347151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fd08-7cd1-4e6c-a021-8b2e9efc1350 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d0ff1eb049e4a50aede95b2f05e94b7, group_id=238633eff27244c58d5dcc731b172500, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:16.348852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fd08-7cd1-4e6c-a021-8b2e9efc1350 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:16.350063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195232-ded9-4443-8df9-6c56956283be None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:16.365299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 213/414] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 06:30:16 2022] PUT /identity/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7/groups/238633eff27244c58d5dcc731b172500/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:30:16.372723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0681-ac44-4e44-a84e-afb9c90b36c2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.373501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0681-ac44-4e44-a84e-afb9c90b36c2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.374029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b0681-ac44-4e44-a84e-afb9c90b36c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.396661 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fe6b0681-ac44-4e44-a84e-afb9c90b36c2 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:16.398521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fe6b0681-ac44-4e44-a84e-afb9c90b36c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:16.418199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:16.459030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195232-ded9-4443-8df9-6c56956283be None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:16.460760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:16.489344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 214/415] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:16.510736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:16.511434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.511722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.512001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.513164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:16.514889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:16.517223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 202/416] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:16 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:16.523031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b0681-ac44-4e44-a84e-afb9c90b36c2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:16.642197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b0681-ac44-4e44-a84e-afb9c90b36c2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:16.683403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:16.684493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:16.684974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:16.685445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121/groups/2e8d6a65ade747789d78a70815caa921/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:16.688216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] RBAC: Authorizing `identity:create_grant(project_id=8823c7322d7b4c53ad080968a7bfe121, group_id=2e8d6a65ade747789d78a70815caa921, role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:16.690277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:16.713293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 215/417] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Aug 24 06:30:16 2022] PUT /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121/groups/2e8d6a65ade747789d78a70815caa921/roles/c37a4d448aa147ebb487e7de2099e1db => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:16.849640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:16.981267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e79f6f-cbab-4633-a753-22a93c4399f7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:17.033868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34a2c7-e5f7-48d0-bcb9-07e301fc6577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:17.034524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a2c7-e5f7-48d0-bcb9-07e301fc6577 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:17.034806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a2c7-e5f7-48d0-bcb9-07e301fc6577 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:17.035076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34a2c7-e5f7-48d0-bcb9-07e301fc6577 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:17.049510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34a2c7-e5f7-48d0-bcb9-07e301fc6577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ee34a2c7-e5f7-48d0-bcb9-07e301fc6577 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:17.054267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:17.055145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:17.055989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:17.059305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 203/418] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:17.064657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:17.082916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:17.168922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9038c-cb66-489f-90ec-b44f9fa35fcc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:17.169245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9038c-cb66-489f-90ec-b44f9fa35fcc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:17.169475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9038c-cb66-489f-90ec-b44f9fa35fcc None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:17.170977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 204/419] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:17.177748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3ce19-6b30-4023-81d5-7934f97f959e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:17.178044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3ce19-6b30-4023-81d5-7934f97f959e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:17.178903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3ce19-6b30-4023-81d5-7934f97f959e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:17.211450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-75d3ce19-6b30-4023-81d5-7934f97f959e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:17.212657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-75d3ce19-6b30-4023-81d5-7934f97f959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:17.222704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887ee3-7928-41c9-8832-1e6f020b9081 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:17.254856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:17.286852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:17.287503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:17.287785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:17.288058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:17.288737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:17.289040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:17.290436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:17.308772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 216/420] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:17.315533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:17.335513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 205/421] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:17.404187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:17.448158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790e5c-e868-4f02-97a6-0db6cad6c1ad None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:17.482686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:17.483393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:17.483672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:17.484140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:17.554646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:17.556054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:17.628617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:17.662178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 217/422] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:17.672022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3ce19-6b30-4023-81d5-7934f97f959e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:17.674402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:17.774508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff9263-f9cf-4110-94d0-6dfd21726e6a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:17.821197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:17.821874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:17.822161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:17.822443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:17.827207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3ce19-6b30-4023-81d5-7934f97f959e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:17.833124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:17.837843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:17.838733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:17.840449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 218/423] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:30:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:17.852087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:17.870403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:17.871060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:17.871373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:17.871650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:17.875072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:17.876574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:17.888137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 206/424] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:17 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:17.890651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:17.932612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:17.933268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:18.014009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb57f74-fd56-4ff8-8767-11e326903104 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:18.051996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:18.052922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.053206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.053482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.054756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:18.056280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:18.058182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 207/425] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:17 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:18.065297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:18.142861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e62adc-e78b-4b02-b438-21be1daafb92 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:18.171075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:18.178518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9911962-ed35-48eb-b232-ab3418baceb7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:18.215429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:18.216526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.217036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.217407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e268b2-35df-4bec-961a-318d51960048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:18.217719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.218664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e268b2-35df-4bec-961a-318d51960048 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.218782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e268b2-35df-4bec-961a-318d51960048 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.219063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e268b2-35df-4bec-961a-318d51960048 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.219572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:18.219840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e268b2-35df-4bec-961a-318d51960048 None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:18.220149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e268b2-35df-4bec-961a-318d51960048 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:18.220314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:18.221582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e268b2-35df-4bec-961a-318d51960048 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:18.222017 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:18.228519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 208/426] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:18 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:18.235855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438ab4-5fad-4852-9b8a-bb2c23258841 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.235855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438ab4-5fad-4852-9b8a-bb2c23258841 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.235855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438ab4-5fad-4852-9b8a-bb2c23258841 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.235855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 209/427] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:18.238252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fe5fb-600e-4e0e-a9b8-bb58f716a4da None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.238611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fe5fb-600e-4e0e-a9b8-bb58f716a4da None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.238955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fe5fb-600e-4e0e-a9b8-bb58f716a4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.240641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 219/428] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:18.247877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:18.276856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4f2fe5fb-600e-4e0e-a9b8-bb58f716a4da None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:18.279802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4f2fe5fb-600e-4e0e-a9b8-bb58f716a4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:18.417327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 210/429] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:18.429799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:18.437386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3eb7d-b700-4b8c-a6f7-6278b4d45e1f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:18.473576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:18.474197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.474478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.474748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.546127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:18.547710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:18.648315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:18.656844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 220/430] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:30:18.662578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fe5fb-600e-4e0e-a9b8-bb58f716a4da None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:18.729690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:18.775975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:18.788665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fe5fb-600e-4e0e-a9b8-bb58f716a4da None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:18.826407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a656d06-342f-40b7-97bf-4114d998c2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:18.827301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a656d06-342f-40b7-97bf-4114d998c2cf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.827431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a656d06-342f-40b7-97bf-4114d998c2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.827688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a656d06-342f-40b7-97bf-4114d998c2cf None admin] PATH_INFO: `/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.828857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a656d06-342f-40b7-97bf-4114d998c2cf None admin] RBAC: Authorizing `identity:get_project(project_id=c55677a9e1e64d8a8fbbf23b820b8ee9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:18.830308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a656d06-342f-40b7-97bf-4114d998c2cf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:18.832016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 211/431] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:18 2022] GET /identity/v3/projects/c55677a9e1e64d8a8fbbf23b820b8ee9 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:18.833172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38052f2a-27bd-4c90-b5e7-2e0b305b776c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.833450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38052f2a-27bd-4c90-b5e7-2e0b305b776c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.833728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38052f2a-27bd-4c90-b5e7-2e0b305b776c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.835060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 212/432] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:18.840509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38052f2a-27bd-4c90-b5e7-2e0b305b776c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:18.853680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095ba9-dc95-4ade-a8ca-6d9e3ab6cd6e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:18.902069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393c8a9-d5c3-46d3-a241-ff2b61b235d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:18.906291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393c8a9-d5c3-46d3-a241-ff2b61b235d4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.906291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393c8a9-d5c3-46d3-a241-ff2b61b235d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.906291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393c8a9-d5c3-46d3-a241-ff2b61b235d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.917328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393c8a9-d5c3-46d3-a241-ff2b61b235d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4393c8a9-d5c3-46d3-a241-ff2b61b235d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:18.925050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:18.925984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 221/433] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:18.926825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9b7f-8be3-4bd9-b480-a48be21dd36d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:18.928102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9b7f-8be3-4bd9-b480-a48be21dd36d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:18.928102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9b7f-8be3-4bd9-b480-a48be21dd36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:18.947079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-63ed9b7f-8be3-4bd9-b480-a48be21dd36d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:18.948431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-63ed9b7f-8be3-4bd9-b480-a48be21dd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:18.971087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38052f2a-27bd-4c90-b5e7-2e0b305b776c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.016573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.017590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.018247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.018447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.018952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.020296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:19.021740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:19.024453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 213/434] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:18 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:19.025278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.045598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 222/435] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:19.047303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed9b7f-8be3-4bd9-b480-a48be21dd36d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.061273 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.141784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf9141-0a14-4d09-aca9-45d90804648d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.182409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599c53a-868b-4817-832e-937d8ad0a916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.183294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599c53a-868b-4817-832e-937d8ad0a916 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.183729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599c53a-868b-4817-832e-937d8ad0a916 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.184024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599c53a-868b-4817-832e-937d8ad0a916 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.184650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599c53a-868b-4817-832e-937d8ad0a916 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:19.184948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599c53a-868b-4817-832e-937d8ad0a916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:19.186325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599c53a-868b-4817-832e-937d8ad0a916 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:19.189545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed9b7f-8be3-4bd9-b480-a48be21dd36d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.195261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 214/436] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:19.201274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f922c3-d863-4567-980a-c2b362679490 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.201566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f922c3-d863-4567-980a-c2b362679490 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.202030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f922c3-d863-4567-980a-c2b362679490 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.203644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 215/437] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:19.208355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f922c3-d863-4567-980a-c2b362679490 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.231937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16440e-60cc-4bfc-a3eb-1f3ab7910aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.232594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16440e-60cc-4bfc-a3eb-1f3ab7910aec None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.232833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16440e-60cc-4bfc-a3eb-1f3ab7910aec None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.233133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16440e-60cc-4bfc-a3eb-1f3ab7910aec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.233952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16440e-60cc-4bfc-a3eb-1f3ab7910aec None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:19.234258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16440e-60cc-4bfc-a3eb-1f3ab7910aec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:19.235771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16440e-60cc-4bfc-a3eb-1f3ab7910aec None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:19.243068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 223/438] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=c55677a9e1e64d8a8fbbf23b820b8ee9&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:19.248703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfdfdc-4569-43b9-84b9-28565c919758 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.249368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfdfdc-4569-43b9-84b9-28565c919758 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.250133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfdfdc-4569-43b9-84b9-28565c919758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.276744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ddbfdfdc-4569-43b9-84b9-28565c919758 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:19.279184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ddbfdfdc-4569-43b9-84b9-28565c919758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:19.316311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.334979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f922c3-d863-4567-980a-c2b362679490 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.337705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.365409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 224/439] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:19.366255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfdfdc-4569-43b9-84b9-28565c919758 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.392058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.393046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.393456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.394045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.476534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:19.478120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:19.494780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfdfdc-4569-43b9-84b9-28565c919758 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.537459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.538166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.538449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.538720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.558248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:19.564163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:19.565113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:19.565963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 225/440] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:19.571077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.594653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.609308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 216/441] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:19.615321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.626664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.695622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668670b3-b105-45af-a5f2-f7630c1d508f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.722326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6628f3-6aeb-48f4-a037-92300b20fe09 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.732798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.733444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.733726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.733999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.734674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:19.734979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:19.736408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:19.744576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 226/442] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:19.745272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.760300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.760936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.761214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.761484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.762581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:19.764039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:19.765792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 217/443] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:19.766549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.822413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.846990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:19.850240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78470155-3e77-4e2a-9de0-8f49bc82103f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.868011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56898b-e6c3-43f9-80e3-940ff445b0ae None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:19.885847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.886491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.886775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.887053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.897557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:19.903492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:19.904397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:19.904497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:19.905513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:19.906325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:19.906951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:19.906951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:19.907108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 227/444] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:19.907108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:19.976935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:19.978413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.007444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de360f7a-4da3-462d-8aab-44c3c577646f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.032047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.041433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.042072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.042358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.042635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.043868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.045330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.047026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 228/445] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:20.047817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.076734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.078531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 218/446] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:20.079540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.149866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bd7fe-1294-4c8c-b80d-85736c043bf0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.177609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbe466-bef8-41a3-8819-6d613d80cb28 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.184968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.185585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.185872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.186152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.186860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:20.187289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.189057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.197074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 229/447] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:20.197705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.216678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.217306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.217623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.217899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.218644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=ffb20dd7d8254b1b960acfe117faa5c3, scope.project.id=8823c7322d7b4c53ad080968a7bfe121) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:20.218959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.220457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.226546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 219/448] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=ffb20dd7d8254b1b960acfe117faa5c3 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:20.227354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.258576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.297839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.315220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208aba8b-f382-409a-a757-f6a6266b2a1a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.349448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2042e6-390e-41fe-8ac8-5fe5d670c480 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.354971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.356012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.356498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.357486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] PATH_INFO: `/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.358980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] RBAC: Authorizing `identity:get_project(project_id=4d0ff1eb049e4a50aede95b2f05e94b7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.361539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.363715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 230/449] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/projects/4d0ff1eb049e4a50aede95b2f05e94b7 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:20.369429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.392415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.393056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.393341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.393619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.478992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.480576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.487762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed5d58-475f-4195-aa88-df9e9726dae2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.507142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.531789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.532439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.532720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.533000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] PATH_INFO: `/v3/groups/238633eff27244c58d5dcc731b172500` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.535002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] RBAC: Authorizing `identity:get_group(group_id=238633eff27244c58d5dcc731b172500)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.536925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.538500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 231/450] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/groups/238633eff27244c58d5dcc731b172500 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:20.544661 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.578531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.597337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 220/451] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:20.603253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.661208 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff78f6-c669-409b-99d0-ceb6bafc3930 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.695835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.696488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.696715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.696999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.697743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] RBAC: Adding query filter params (group.id=238633eff27244c58d5dcc731b172500, role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=4d0ff1eb049e4a50aede95b2f05e94b7) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:20.698052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.699556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.702968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b3006-2dd0-4521-9bc7-af0ccf38b88e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.705384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 232/452] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/role_assignments?group.id=238633eff27244c58d5dcc731b172500&scope.project.id=4d0ff1eb049e4a50aede95b2f05e94b7&role.id=c64ac97b1795434390dc18ab162aec39 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:20.725402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.741113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.741818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.742100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.742370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] PATH_INFO: `/v3/projects/8823c7322d7b4c53ad080968a7bfe121` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.743575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] RBAC: Authorizing `identity:get_project(project_id=8823c7322d7b4c53ad080968a7bfe121)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.745026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.746699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 221/453] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/projects/8823c7322d7b4c53ad080968a7bfe121 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:20.752727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:20.852203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:20.863010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4db3b-1ed3-4b0c-b4e0-ec74a67dcee6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:20.898738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe00872-77b3-4e16-ad11-0272265ab5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:20.899405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe00872-77b3-4e16-ad11-0272265ab5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:20.899690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe00872-77b3-4e16-ad11-0272265ab5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:20.899961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe00872-77b3-4e16-ad11-0272265ab5a5 None admin] PATH_INFO: `/v3/groups/2e8d6a65ade747789d78a70815caa921` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:20.901274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe00872-77b3-4e16-ad11-0272265ab5a5 None admin] RBAC: Authorizing `identity:get_group(group_id=2e8d6a65ade747789d78a70815caa921)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:20.902744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe00872-77b3-4e16-ad11-0272265ab5a5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:20.904354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 233/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/groups/2e8d6a65ade747789d78a70815caa921 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:20.910796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:21.020327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883c360-82e2-414c-a475-a39577ffc25f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:21.062257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a202b-74f4-424e-a9f2-60536a49be0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:21.062798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a202b-74f4-424e-a9f2-60536a49be0a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:21.063079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a202b-74f4-424e-a9f2-60536a49be0a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:21.063385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a202b-74f4-424e-a9f2-60536a49be0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:21.064480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a202b-74f4-424e-a9f2-60536a49be0a None admin] RBAC: Adding query filter params (group.id=2e8d6a65ade747789d78a70815caa921, role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=8823c7322d7b4c53ad080968a7bfe121) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:21.064792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a202b-74f4-424e-a9f2-60536a49be0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:21.066237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a202b-74f4-424e-a9f2-60536a49be0a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:21.072155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 222/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Aug 24 06:30:20 2022] GET /identity/v3/role_assignments?group.id=2e8d6a65ade747789d78a70815caa921&scope.project.id=8823c7322d7b4c53ad080968a7bfe121&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:23.481687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac702f-d0b0-471d-a653-898cdc18c882 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:23.482189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac702f-d0b0-471d-a653-898cdc18c882 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:23.482189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac702f-d0b0-471d-a653-898cdc18c882 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:23.484088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 234/456] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:23.489747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b7e74-2e5d-4ab0-ba97-1a65a7bd5ab9 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:23.490203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b7e74-2e5d-4ab0-ba97-1a65a7bd5ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:23.490278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b7e74-2e5d-4ab0-ba97-1a65a7bd5ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:23.520252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6f0b7e74-2e5d-4ab0-ba97-1a65a7bd5ab9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:23.521485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6f0b7e74-2e5d-4ab0-ba97-1a65a7bd5ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:23.581696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:23.605081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 223/457] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:23.761016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1fa95-cd6d-4593-aa74-43be0de3c609 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:23.761678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1fa95-cd6d-4593-aa74-43be0de3c609 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:23.762172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1fa95-cd6d-4593-aa74-43be0de3c609 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:23.764070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 235/458] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:23.769354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb7575-4e86-47d6-a0fe-23afee48f848 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:23.769650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb7575-4e86-47d6-a0fe-23afee48f848 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:23.769933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb7575-4e86-47d6-a0fe-23afee48f848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:23.800226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3edb7575-4e86-47d6-a0fe-23afee48f848 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:23.801448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3edb7575-4e86-47d6-a0fe-23afee48f848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:23.880283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 224/459] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:23.935175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05166c1-c137-47d0-821e-9c5f3c744354 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:23.935513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05166c1-c137-47d0-821e-9c5f3c744354 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:23.935750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05166c1-c137-47d0-821e-9c5f3c744354 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:23.937614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 236/460] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:23.942944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a03de6-0e53-4477-8cf7-3570f3ad7743 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:23.943267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a03de6-0e53-4477-8cf7-3570f3ad7743 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:23.943552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a03de6-0e53-4477-8cf7-3570f3ad7743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:23.961040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05166c1-c137-47d0-821e-9c5f3c744354 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:23.968062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-34a03de6-0e53-4477-8cf7-3570f3ad7743 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:23.969317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-34a03de6-0e53-4477-8cf7-3570f3ad7743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:24.076950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05166c1-c137-47d0-821e-9c5f3c744354 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:24.104958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 225/461] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:24.132814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:24.133465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.133748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.134025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.146870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:24.148697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:24.172962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:24.173924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:24.174799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:24.174799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:24.174799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:24.176600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 237/462] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:24.182848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a03de6-0e53-4477-8cf7-3570f3ad7743 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:24.260720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:24.264073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:24.292869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:24.342214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a03de6-0e53-4477-8cf7-3570f3ad7743 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:24.396257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:24.397710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.400640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.401098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.402801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:24.404263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:24.406115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:24.429899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 226/463] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:24.434137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:24.468902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee5290-9fb8-4bfe-b49d-79eac663b4f0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:24.545586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:24.546521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.546808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.547081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.563517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:24.567140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:24.585961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:24.587961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:24.589087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:24.589087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:24.589087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:24.589535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 238/464] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:24.590590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:24.612150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b3b2d-65bb-4c04-b3bb-f8a90a66da22 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:24.613649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:24.654310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324c47f-363f-4bde-9bc4-3c7d012a52ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:24.655037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324c47f-363f-4bde-9bc4-3c7d012a52ef None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.655452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324c47f-363f-4bde-9bc4-3c7d012a52ef None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.655802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324c47f-363f-4bde-9bc4-3c7d012a52ef None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.656834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324c47f-363f-4bde-9bc4-3c7d012a52ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:24.658544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324c47f-363f-4bde-9bc4-3c7d012a52ef None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:24.676417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:24.684676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 227/465] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:30:24 2022] POST /identity/v3/users => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:24.686610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c65b9-d4ef-4532-bbf7-6b4bef234f91 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.687096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c65b9-d4ef-4532-bbf7-6b4bef234f91 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.687539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c65b9-d4ef-4532-bbf7-6b4bef234f91 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.694282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 228/466] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:24.695557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c65b9-d4ef-4532-bbf7-6b4bef234f91 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:24.735375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a94ee-26b4-4cad-9006-a9cc8c920ac6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:24.781700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e598fda-ef2f-43c3-902a-420f35388a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:24.782770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e598fda-ef2f-43c3-902a-420f35388a3e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.783352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e598fda-ef2f-43c3-902a-420f35388a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.783817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e598fda-ef2f-43c3-902a-420f35388a3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.797444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e598fda-ef2f-43c3-902a-420f35388a3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:24.799337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e598fda-ef2f-43c3-902a-420f35388a3e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5e598fda-ef2f-43c3-902a-420f35388a3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:24.817699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:24.818657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:24.819542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:24.819542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:24.819542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:24.824232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 239/467] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:24.825707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41765078-5828-4e01-b62c-51f0eebf32b1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.825998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41765078-5828-4e01-b62c-51f0eebf32b1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.826469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41765078-5828-4e01-b62c-51f0eebf32b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.870091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-41765078-5828-4e01-b62c-51f0eebf32b1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:24.872818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41765078-5828-4e01-b62c-51f0eebf32b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:24.916871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c65b9-d4ef-4532-bbf7-6b4bef234f91 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:24.965874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:24.966779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:24.967237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:24.968111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:24.968935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:24.971485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:24.974180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:24.980025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 240/468] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:24.980166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41765078-5828-4e01-b62c-51f0eebf32b1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:24.988201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 229/469] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:24.995568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:25.028068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:25.034414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:25.109172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41765078-5828-4e01-b62c-51f0eebf32b1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:25.140349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1f3e-b1b4-4ada-8203-9e503ba2162a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:25.166770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:25.167669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.168125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.168546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.169346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:25.169795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:25.171597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:25.193454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 241/470] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:25.199354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:25.223295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495887b7-570a-45f9-b60f-82823dffcd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:25.224198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495887b7-570a-45f9-b60f-82823dffcd21 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.224481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495887b7-570a-45f9-b60f-82823dffcd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.224767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495887b7-570a-45f9-b60f-82823dffcd21 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.225574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495887b7-570a-45f9-b60f-82823dffcd21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:25.227363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495887b7-570a-45f9-b60f-82823dffcd21 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:25.257173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 230/471] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:30:24 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:25.258899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cc338-5a1f-41f0-bcea-0e5e561292c0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.259752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cc338-5a1f-41f0-bcea-0e5e561292c0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.260085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cc338-5a1f-41f0-bcea-0e5e561292c0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.267348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 231/472] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:25.268887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d759a-c765-456b-95be-e0a013e25bc9 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.271328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d759a-c765-456b-95be-e0a013e25bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.271873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d759a-c765-456b-95be-e0a013e25bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.301779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e66d759a-c765-456b-95be-e0a013e25bc9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:25.303073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e66d759a-c765-456b-95be-e0a013e25bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:25.342749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b4100-8776-4c75-bbfc-689df74bc108 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:25.345836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:25.391840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:25.392685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.393068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.393464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.393988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 232/473] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:25.394370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:25.395305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d759a-c765-456b-95be-e0a013e25bc9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:25.396092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:25.399318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:25.428872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 242/474] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:30:25 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:30:25.517599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d759a-c765-456b-95be-e0a013e25bc9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:25.558762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:25.559751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.560536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.560945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.573617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:25.575403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:25.592276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:25.593682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:25.594740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:25.594740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:25.594740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:25.596337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 233/475] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:25.603525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:25.700588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:25.718265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7848f6-2217-482a-a25c-afff4fd0c9c5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:25.761859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:25.763027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.763375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.763649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.764295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:25.764645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:25.766100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:25.768075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:25.780001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 243/476] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:25.786369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:25.813000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:25.825936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150f772-1d64-447f-a4a2-10be9c0b9c44 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:25.874027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:25.875050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.875537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.875987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.876856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:25.878435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:25.892242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.sql.core [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 06:30:25.892242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 06:30:25.892242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [parameters: {'id': '6e864833d4b241948c971b5a4f395c8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 06:30:25.892242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81768) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 06:30:25.911232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b1b06-3905-4d87-9828-0d377b939419 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result.read() Aug 24 06:30:25.918632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:30:25.919610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application yield resource Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise self.value Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application session.commit() Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 06:30:25.920528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.session.flush() Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application compat.raise_( Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 06:30:25.921755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:30:25.922651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result.read() Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:30:25.931974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [parameters: {'id': '6e864833d4b241948c971b5a4f395c8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:25.932895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 06:30:25.933947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:25.936524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 234/477] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 06:30:25 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:25.942901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:25.963247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:25.963725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:25.967599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:25.968042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:25.968811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:25.970565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:26.031792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 244/478] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:30:25 2022] POST /identity/v3/users => generated 249 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:30:26.062769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5d27c-8555-435c-b041-21be8a407a6a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:26.086309 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:26.106450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15faa29-db4d-4eae-a989-310a63b63d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:26.107413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15faa29-db4d-4eae-a989-310a63b63d5d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:26.107833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15faa29-db4d-4eae-a989-310a63b63d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:26.108316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15faa29-db4d-4eae-a989-310a63b63d5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:26.125766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15faa29-db4d-4eae-a989-310a63b63d5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d15faa29-db4d-4eae-a989-310a63b63d5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:26.131674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:26.132882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:30:26.133917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:26.136589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 235/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:30:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:26.142773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:26.261345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:26.293989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1cfe-a77e-4b0e-a898-cf85fbf60788 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:26.344616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a480e-4538-487d-84e2-88f5ecb19629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:26.345252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a480e-4538-487d-84e2-88f5ecb19629 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:26.345532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a480e-4538-487d-84e2-88f5ecb19629 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:26.345802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a480e-4538-487d-84e2-88f5ecb19629 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:26.346493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a480e-4538-487d-84e2-88f5ecb19629 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:26.346867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a480e-4538-487d-84e2-88f5ecb19629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:26.348310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a480e-4538-487d-84e2-88f5ecb19629 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:26.359839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 245/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:27.075778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3438486-ff76-4245-ba03-5b7a9b0a97d3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:27.076815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3438486-ff76-4245-ba03-5b7a9b0a97d3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:27.077354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3438486-ff76-4245-ba03-5b7a9b0a97d3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:27.079167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 236/481] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:27.084824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b1818-0691-49a1-8c7a-aaf7b146938c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:27.085635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b1818-0691-49a1-8c7a-aaf7b146938c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:27.086113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b1818-0691-49a1-8c7a-aaf7b146938c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:27.104951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-948b1818-0691-49a1-8c7a-aaf7b146938c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:27.106523 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-948b1818-0691-49a1-8c7a-aaf7b146938c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:27.178467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 246/482] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:27.478774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3438486-ff76-4245-ba03-5b7a9b0a97d3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:27.599965 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3438486-ff76-4245-ba03-5b7a9b0a97d3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:27.649811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:27.650839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:27.651358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:27.651829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:27.666463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:27.672070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:27.673089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:27.673899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:27.676612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 237/483] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:27.682149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b1818-0691-49a1-8c7a-aaf7b146938c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:27.767722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:27.811773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b1818-0691-49a1-8c7a-aaf7b146938c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:27.852213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29a1e1-76bc-40b0-afe9-cfe761a10947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:27.853190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a1e1-76bc-40b0-afe9-cfe761a10947 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:27.853698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a1e1-76bc-40b0-afe9-cfe761a10947 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:27.854165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a1e1-76bc-40b0-afe9-cfe761a10947 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:27.855047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29a1e1-76bc-40b0-afe9-cfe761a10947 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:27.855580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29a1e1-76bc-40b0-afe9-cfe761a10947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:27.857197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29a1e1-76bc-40b0-afe9-cfe761a10947 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:27.870117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 247/484] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:27.875971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:27.916182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02788-8154-4d9e-a0cb-27a60411083f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:27.916503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02788-8154-4d9e-a0cb-27a60411083f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:27.916748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02788-8154-4d9e-a0cb-27a60411083f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:27.918355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 248/485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:27.923963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c3943-6458-4867-abef-c314ac81b6fb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:27.924247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c3943-6458-4867-abef-c314ac81b6fb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:27.924528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c3943-6458-4867-abef-c314ac81b6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:27.933351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:27.959225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-744c3943-6458-4867-abef-c314ac81b6fb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:27.959225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-744c3943-6458-4867-abef-c314ac81b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:28.013125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27115e59-0586-40f9-8333-c6350faf219f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.048795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.049404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.049685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.049962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.061892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:28.064271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:28.080313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:28.081720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:28.082577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:28.082577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:28.082577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:28.082577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 238/486] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:28.089556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.106549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 249/487] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:28.171552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9290d-b503-4e9e-8454-ba82d96512b3 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.171843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9290d-b503-4e9e-8454-ba82d96512b3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.172102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9290d-b503-4e9e-8454-ba82d96512b3 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.177440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 250/488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:30:28.179022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb16e6-f9bd-4741-8e7e-6256c65554a9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.179339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb16e6-f9bd-4741-8e7e-6256c65554a9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.179621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb16e6-f9bd-4741-8e7e-6256c65554a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.207990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-debb16e6-f9bd-4741-8e7e-6256c65554a9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:28.211566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-debb16e6-f9bd-4741-8e7e-6256c65554a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:28.219850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:28.238114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8179186-be72-47d4-a55f-bd85c5260420 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.311307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.313399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.314345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.314943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.318470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:28.318992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:28.320627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:28.325952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:28.345972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 239/489] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:28.351814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.351946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 251/490] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:28.432334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb16e6-f9bd-4741-8e7e-6256c65554a9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.463331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a6c42-b7b9-4e03-8fa0-2d6191e878e2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.517872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.518771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.519208 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.519628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.539245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:28.541205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:28.560335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:28.561624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:28.562798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:28.570497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb16e6-f9bd-4741-8e7e-6256c65554a9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.572027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 240/491] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:28.578045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.596725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:28.607822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.608599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.608990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.609296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.628312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:28.633091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:28.633975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:28.634792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 252/492] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:28.640236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.698073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4afb62-5b35-4fc8-aa32-80c87f995b73 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.712350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:28.739383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.740473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.740760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.741033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.741782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:28.742090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:28.743569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:28.765055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17d1fd-463c-453e-90bf-e94cba8ba15e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.785769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 241/493] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:28.786459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.801315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca9d9-ccb1-46c3-83bd-8d2802e3e261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.801936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca9d9-ccb1-46c3-83bd-8d2802e3e261 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.802217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca9d9-ccb1-46c3-83bd-8d2802e3e261 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.802489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca9d9-ccb1-46c3-83bd-8d2802e3e261 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.803181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca9d9-ccb1-46c3-83bd-8d2802e3e261 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:28.803596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca9d9-ccb1-46c3-83bd-8d2802e3e261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:28.804994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca9d9-ccb1-46c3-83bd-8d2802e3e261 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:28.818402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 253/494] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:28.818402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9ca9-7d22-48b8-8e87-0a471fb04747 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.818402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9ca9-7d22-48b8-8e87-0a471fb04747 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.818402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9ca9-7d22-48b8-8e87-0a471fb04747 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.818402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 254/495] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:28.818402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd9ca9-7d22-48b8-8e87-0a471fb04747 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.867472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:28.898093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd3a9c-3723-4fdf-99d7-15d1c04a985e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.924687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd9ca9-7d22-48b8-8e87-0a471fb04747 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:28.931941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.932571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.932851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.933129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:28.937733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:28.943682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:28.948403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:28.949300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:28.950110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 242/496] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:28.950781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:28.966258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035e5d6-47b2-4004-a8e5-35f26f8e52cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:28.966878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035e5d6-47b2-4004-a8e5-35f26f8e52cc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:28.967163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035e5d6-47b2-4004-a8e5-35f26f8e52cc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:28.967470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035e5d6-47b2-4004-a8e5-35f26f8e52cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.035455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035e5d6-47b2-4004-a8e5-35f26f8e52cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.036926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035e5d6-47b2-4004-a8e5-35f26f8e52cc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.051558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6a434-61e4-4a99-95a7-13dd0ad09ab7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.081470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.087976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc34fc-1c2a-483a-9f01-dd893b358639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.088608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc34fc-1c2a-483a-9f01-dd893b358639 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.088887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc34fc-1c2a-483a-9f01-dd893b358639 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.089161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc34fc-1c2a-483a-9f01-dd893b358639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.098985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc34fc-1c2a-483a-9f01-dd893b358639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.100477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc34fc-1c2a-483a-9f01-dd893b358639 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-adfc34fc-1c2a-483a-9f01-dd893b358639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:29.116013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:29.116981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:29.117823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:29.117823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.117823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:29.117823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 243/497] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:29.118553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aadfce-6cb8-4aab-b72e-62837118b356 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.118834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aadfce-6cb8-4aab-b72e-62837118b356 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.119110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aadfce-6cb8-4aab-b72e-62837118b356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.137396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-83aadfce-6cb8-4aab-b72e-62837118b356 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:29.138609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-83aadfce-6cb8-4aab-b72e-62837118b356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:29.143396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 255/498] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:29.144683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba4cd-3254-4d7b-ad19-8df24d477fdf None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.144973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba4cd-3254-4d7b-ad19-8df24d477fdf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.145250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba4cd-3254-4d7b-ad19-8df24d477fdf None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.146495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 256/499] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:29.147234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba4cd-3254-4d7b-ad19-8df24d477fdf None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.162327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.208180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 244/500] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:29.209478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aadfce-6cb8-4aab-b72e-62837118b356 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.257703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba4cd-3254-4d7b-ad19-8df24d477fdf None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.292172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.292783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.293066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.293339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.294019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:29.294325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.295764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.308918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 257/501] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:29.308918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.327690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aadfce-6cb8-4aab-b72e-62837118b356 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.341548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.364510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d3702-3fa7-44ac-9114-87898d451215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.365126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3702-3fa7-44ac-9114-87898d451215 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.365409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3702-3fa7-44ac-9114-87898d451215 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.365684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3702-3fa7-44ac-9114-87898d451215 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.366486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d3702-3fa7-44ac-9114-87898d451215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:29.366627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d3702-3fa7-44ac-9114-87898d451215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.368161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d3702-3fa7-44ac-9114-87898d451215 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.384668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 245/502] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:29.384668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c8ba5-65c2-4669-85f5-6fe050ce05ad None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.384668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c8ba5-65c2-4669-85f5-6fe050ce05ad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.384668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c8ba5-65c2-4669-85f5-6fe050ce05ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.390128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.403129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9b0c8ba5-65c2-4669-85f5-6fe050ce05ad None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:29.404484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9b0c8ba5-65c2-4669-85f5-6fe050ce05ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:29.416080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5025aca-5c26-45f8-9eb5-7e89ade0e85d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.451092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.451793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.452124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.452554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.463873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.465364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.475918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 246/503] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:29.476766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c8ba5-65c2-4669-85f5-6fe050ce05ad None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:29.480584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:29.481500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:29.482396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:29.482396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.482396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:29.483369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 258/504] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:29.484881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.580561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c8ba5-65c2-4669-85f5-6fe050ce05ad None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.582097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.603170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1533e-0d65-4933-bf0a-64673aa6b805 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.618084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.619100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.619538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.619821 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.630200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.630882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.631715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.641373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.642514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.642900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.643181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:29.648226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:29.649134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:29.649996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:29.649996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.649996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:29.651490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.652949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.654217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 247/505] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:29.654857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:29.666099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:29.667215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 259/506] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:29.672231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.766374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c834f6-7d5b-4962-95ce-cb31cc7921c7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.771260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1c8d-4b83-4168-962c-baac90ffdf24 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.802098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.802819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.803097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.803397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.805728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.806651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.806940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.807227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.807877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:29.808184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.809573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.815582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.817052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.823162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 260/507] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:29.823995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:29.833253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:29.834327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:29.835288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:29.835288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:29.835288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:29.835288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 248/508] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:29.835515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.849014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.876468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:29.929013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cf5f9-9c4d-4dd5-812d-27fddfd5ff15 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.945394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a288302-8253-43e4-b6c6-c0a3480a849c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:29.963067 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.963782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.964070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.964349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.965003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:29.965315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.966733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:29.979719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 261/509] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:29.980567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:29.985456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:29.986090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:29.986373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:29.986642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:29.987420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:29.987722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:29.989172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.007428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 249/510] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:30.008392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.077713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.104958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeeee02-97e7-421d-b313-307d55759e34 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.112252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.115888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0092e-471c-4b52-84e8-46fec2bcd7fa None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.140248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.140910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.141200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.141475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.149529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.150161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.150444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.150720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.153618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:30.163882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:30.165016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:30.165983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:30.167008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.168592 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 262/511] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:30.168592 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.168787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:30.183283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:30.184333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 250/512] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:30.190567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.286693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e000c-4fbf-4476-a968-25ef9d1cacd2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.309870 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda04e43-7c46-4299-95f4-b2bc4cb8fa13 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.326262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.326882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.327291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.327511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.328264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:30.328574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.329983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.334435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.343015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 263/513] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:30.344000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.351957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.353131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.353131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.353131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.364166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.365780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.378439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:30.379735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:30.381233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:30.382226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 251/514] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:30.384405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.458241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687b948-7afa-4168-8d9c-f2c359f9812e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.494586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.495356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.495575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.495803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.497781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903aa6d8-d863-49cf-be40-67f0f7bf9db3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.531898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.532630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.532980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.533260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.533932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:30.534242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.535675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.543616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 252/515] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:30.544405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.568772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.570271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.592174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.599252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.651289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668278a7-04fe-4bca-961d-468ebc9286a8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.680824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 264/516] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:30.681656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.685240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.685940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.686225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.686884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.687634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:30.687935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.689766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.708432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 253/517] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:30.709180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.788554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b4bc2-98b1-4499-a901-7ea08ddccccd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.815245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee23b6-efaa-4fca-8c05-5530d355ef03 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:30.815453 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.823274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:30.824335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.824991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.825301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.825633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.826308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.828027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.849174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 265/518] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:30:30 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:30.850005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.863002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:30.864060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:30.864464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:30.864704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:30.865414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:30.865731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:30.867580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:30.891081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 254/519] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:30.891837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:30.978800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a068d8-32dc-4373-a491-96f61fb50440 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.017325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2cea9-37fb-4dd3-a90c-bd9ff34ea209 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.022041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.022910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.023337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.023602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.038990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.040551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:31.059374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:31.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:31.061158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:31.061158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:31.061158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:31.061158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 266/520] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:31.062058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.069158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.069822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.070109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.070386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.081404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.082901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:31.097799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:31.098755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:31.099934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:31.099934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:31.099934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:31.099934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 255/521] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:31.099934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.106803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.119553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.186274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda50bb-533c-47fa-bbd9-70549146f594 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.206988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68ed71-b40d-471e-b22b-49184b46bc6f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.230161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.230714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.230999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.231297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.232050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=8e1ec869263246f58f5a32d855a3d302) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:31.232882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.235008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.242265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 267/522] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/role_assignments?user.id=8e1ec869263246f58f5a32d855a3d302&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:31.243053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.256970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.258788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.259640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.260146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.360088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.361973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.367694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.369883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b370ca-6b2c-42a3-8ca9-08195b008cb0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.385557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.413317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.414322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.415241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.415724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.416903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:31.417316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.419595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.437193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 268/523] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:31.438517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.482078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 256/524] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:31.482878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.571919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeab59a-2719-460a-9ec4-b6486d6e85cb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.599217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a68c-bf37-49fb-9bab-47be9ee7a19d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.619800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.629199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.630013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.630709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.631143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.632497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:31.633047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.635228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.646469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.647111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.647438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.647713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.653693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 269/525] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:31.654945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.660391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.661405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:31.673897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:31.674866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:30:31.675834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:31.679527 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 257/526] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:31.680356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.681266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.782241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6430c31-66a6-4016-9509-729f4f555127 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.786351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be363e7-1023-4ca6-bad4-01c9a17ee994 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.820113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.820919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.821275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.821624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.829675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:31.830678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:31.831174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:31.831647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:31.834780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.836356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.839843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:31.841452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:31.852220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:31.853185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:31.854061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:31.854061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:31.854061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:31.854061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 270/527] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:31.854870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:31.856617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:31.857440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:31.858544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:31.860668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 258/528] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:31.861637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:31.903263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.933599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:31.968334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40062490-eb60-46bb-80d3-704062cd283a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:31.979246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299453ee-8b8b-4206-a88d-9c1c99159066 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.005105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.005731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.006018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.006298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.007053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:32.007389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.008858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.015604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.016354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.016733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.017150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.017904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:32.018305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.019901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.043290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 259/529] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:32.043290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:32.049636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 271/530] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 24 06:30:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:32.050490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:32.150834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:32.158450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58d36b-0015-4dbb-8ebf-bc1a9692feb8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.158938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9ffd5-da39-424b-a982-a3ca3e78ebfc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.186191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:32.196652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.197270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.197365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.197574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.197837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.198572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.199058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.199563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.200454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:32.200970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.202701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.204920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.206388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.218699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 260/531] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:32.219920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:32.224605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:32.225442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:32.226345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 272/532] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:32.232321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:32.336856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb88bd-4496-4dde-bc28-b89dab81d801 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.340591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdca43-24ea-45fa-b8ec-a509fa7417bc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.381039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.381678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.381961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.382240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.385486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.386103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.386389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.386656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.387733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:32.388040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.390638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.417612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:32.418033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 273/533] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:32.418033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:32.481544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:32.491177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.493631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.543708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90217e15-be3b-4d25-ba7e-975ad1a00468 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.590739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.591259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.591545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.591822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.592574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=380786eb9b7848d59f3bb0007726dd2a) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:32.593142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.594985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.607234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 274/534] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/role_assignments?user.id=380786eb9b7848d59f3bb0007726dd2a&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:32.607234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:32.657126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 261/535] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:32.664058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:32.776741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:32.828123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470ebbf-9621-4553-a2f3-14f083da1f34 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.842522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:32.860256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f6558-e577-4dc6-983e-b37a95acdf58 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:32.867130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6dd5c-00a0-4cf3-b43b-4592f8e8fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.867966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6dd5c-00a0-4cf3-b43b-4592f8e8fc0f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.868441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6dd5c-00a0-4cf3-b43b-4592f8e8fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.868875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6dd5c-00a0-4cf3-b43b-4592f8e8fc0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.869857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6dd5c-00a0-4cf3-b43b-4592f8e8fc0f None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=c216522efef4447caac245392d95b985) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:32.870377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6dd5c-00a0-4cf3-b43b-4592f8e8fc0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.872161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6dd5c-00a0-4cf3-b43b-4592f8e8fc0f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.878682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 275/536] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/role_assignments?user.id=c216522efef4447caac245392d95b985&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:32.880186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4e6e-5e2e-44f7-b958-c3db12a68902 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.880732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4e6e-5e2e-44f7-b958-c3db12a68902 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.881217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4e6e-5e2e-44f7-b958-c3db12a68902 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.882906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 276/537] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:32.888343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760fe77-a7eb-492d-a291-885798da497a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.890559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760fe77-a7eb-492d-a291-885798da497a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.891294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760fe77-a7eb-492d-a291-885798da497a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.918927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:32.919993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:32.920499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:32.921057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:32.933037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:32.934876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:32.952345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:32.953363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:32.954225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:32.954225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:32.954225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:32.954391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f760fe77-a7eb-492d-a291-885798da497a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:32.954391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f760fe77-a7eb-492d-a291-885798da497a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:32.956750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 262/538] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:32.962625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:33.098854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 277/539] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:33.136035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166ba8-fd83-4731-b4e9-a1ea8811e55b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:33.147581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:33.178196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d4b0-1c7d-4654-a26a-4046b87dc574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:33.179663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d4b0-1c7d-4654-a26a-4046b87dc574 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.180163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d4b0-1c7d-4654-a26a-4046b87dc574 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.180645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d4b0-1c7d-4654-a26a-4046b87dc574 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.181490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d4b0-1c7d-4654-a26a-4046b87dc574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:33.181988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d4b0-1c7d-4654-a26a-4046b87dc574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:33.191717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d4b0-1c7d-4654-a26a-4046b87dc574 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:33.210694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 263/540] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:33.218730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760fe77-a7eb-492d-a291-885798da497a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:33.254973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15cb29-ca58-4707-abe4-06e9700c8d28 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.255348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15cb29-ca58-4707-abe4-06e9700c8d28 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.255622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15cb29-ca58-4707-abe4-06e9700c8d28 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.257153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 264/541] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:33.262561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877533-9dde-4a10-917f-98c55cc29a10 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.262844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877533-9dde-4a10-917f-98c55cc29a10 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.263118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb877533-9dde-4a10-917f-98c55cc29a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.286696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bb877533-9dde-4a10-917f-98c55cc29a10 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:33.288480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bb877533-9dde-4a10-917f-98c55cc29a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:33.387485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 265/542] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:33.477165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760fe77-a7eb-492d-a291-885798da497a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:33.486435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:33.491987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb877533-9dde-4a10-917f-98c55cc29a10 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:33.545242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:33.546053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.546483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.547524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.559978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:33.561508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:33.585563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:33.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:33.587802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 278/543] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:33.593094 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:33.697495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a20293-0165-4aba-8122-b89d9b283ef3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:33.712782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:33.718504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb877533-9dde-4a10-917f-98c55cc29a10 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:33.744917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:33.745538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.745819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.746092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.746796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:33.747106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:33.748775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:33.762109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 279/544] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:33.763786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:33.786028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:33.786667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.786947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.787286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.798282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:33.799038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:33.805013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:33.806528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:33.808004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:33.811601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 266/545] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:33.812668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:33.884884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5506dd-e444-43a7-be20-3a8a2109f733 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:33.924123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68431977-a45a-47f8-a198-7ab037e47b2c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:33.925341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:33.926211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.926692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.927162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.938540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:33.944607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:33.945485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:33.946304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:33.954380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 280/546] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:33.955503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:33.961595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:33.962213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:33.962492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:33.962764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:33.963535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=2fc4592ef54b41059fa7767a5ca3aaa3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:33.963853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:33.965296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:33.975526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 267/547] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:33 2022] GET /identity/v3/role_assignments?user.id=2fc4592ef54b41059fa7767a5ca3aaa3&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:33.976432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:34.055336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:34.057148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:34.073193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66297c0-58bd-426d-8839-53131a9231ac None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:34.110747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:34.111595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.111986 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.112384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.113182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:34.113558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:34.115083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:34.124859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 281/548] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:34.131116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:34.260212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd04d9-9d05-4311-8451-b9be1fc34f74 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:34.310655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa688c9e-f1e3-4aaf-96b8-20e7a7f0f2bc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:34.347871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:34.348709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.350930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.350930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.361058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:34.363561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:34.380249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:34.381210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:34.382754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:34.382754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:34.382754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:34.382754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 282/549] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:34.383416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:34.387683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:34.403929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:34.404570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.404859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.405138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.406095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:34.406597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:34.408028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:34.437768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 268/550] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:34.448218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:34.506120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c4d18-4a45-42dd-a552-3a0ffd5b5ef3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:34.576524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:34.577236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.577527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.577797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.578917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:34.579452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:34.580851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:34.615092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 283/551] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:34.621922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:34.648020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:34.739218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91364701-6c9e-48b6-ac5a-64804c52dc60 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:34.755543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:34.819306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a1bf-c913-4627-8a9e-fd973ec3d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:34.819306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a1bf-c913-4627-8a9e-fd973ec3d425 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.819306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a1bf-c913-4627-8a9e-fd973ec3d425 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.819306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a1bf-c913-4627-8a9e-fd973ec3d425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.847218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a1bf-c913-4627-8a9e-fd973ec3d425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:34.847218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a1bf-c913-4627-8a9e-fd973ec3d425 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:34.847476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb9346-7b8b-47b3-8c4a-896603d32099 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-fc38a1bf-c913-4627-8a9e-fd973ec3d425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:34.872063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:34.873164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:34.874235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:34.874235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:34.874235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:34.878242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 269/552] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:34.879626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649b062-05c3-4114-b292-4c04bad1509c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.879916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649b062-05c3-4114-b292-4c04bad1509c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.880194 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649b062-05c3-4114-b292-4c04bad1509c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.885800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 270/553] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:34.886570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649b062-05c3-4114-b292-4c04bad1509c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:34.934977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285996bb-3c01-4f4d-8106-eceef3f3a76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:34.935670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285996bb-3c01-4f4d-8106-eceef3f3a76d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.935956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285996bb-3c01-4f4d-8106-eceef3f3a76d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.936234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285996bb-3c01-4f4d-8106-eceef3f3a76d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.946026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285996bb-3c01-4f4d-8106-eceef3f3a76d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:34.947530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285996bb-3c01-4f4d-8106-eceef3f3a76d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-285996bb-3c01-4f4d-8106-eceef3f3a76d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:34.966751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:34.968020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:34.969052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:34.971991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 284/554] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:34.973505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87b1de-1917-44ee-8049-d0ff459a35a6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:34.973795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87b1de-1917-44ee-8049-d0ff459a35a6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:34.974071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87b1de-1917-44ee-8049-d0ff459a35a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:34.995641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3b87b1de-1917-44ee-8049-d0ff459a35a6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:35.000252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3b87b1de-1917-44ee-8049-d0ff459a35a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:35.111475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 285/555] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:35.112353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87b1de-1917-44ee-8049-d0ff459a35a6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:35.113426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649b062-05c3-4114-b292-4c04bad1509c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:35.176628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c39bd-0500-4f30-b2e0-98bf1b1525f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:35.177624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c39bd-0500-4f30-b2e0-98bf1b1525f5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.178100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c39bd-0500-4f30-b2e0-98bf1b1525f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.178586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c39bd-0500-4f30-b2e0-98bf1b1525f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.179495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c39bd-0500-4f30-b2e0-98bf1b1525f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:35.180012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c39bd-0500-4f30-b2e0-98bf1b1525f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:35.181706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c39bd-0500-4f30-b2e0-98bf1b1525f5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:35.184684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:35.187720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:35.219568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 271/556] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:35.234399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585aab9-4238-4596-a042-c3e07677f6c9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.235047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585aab9-4238-4596-a042-c3e07677f6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.235554 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585aab9-4238-4596-a042-c3e07677f6c9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.239420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 272/557] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:35 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:35.241078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585aab9-4238-4596-a042-c3e07677f6c9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:35.261929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87b1de-1917-44ee-8049-d0ff459a35a6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:35.310038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a6207-f268-4c18-88ee-52ed308b3ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:35.310656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6207-f268-4c18-88ee-52ed308b3ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.310935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6207-f268-4c18-88ee-52ed308b3ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.311564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6207-f268-4c18-88ee-52ed308b3ba1 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.312073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a6207-f268-4c18-88ee-52ed308b3ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:35.312390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a6207-f268-4c18-88ee-52ed308b3ba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:35.314380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a6207-f268-4c18-88ee-52ed308b3ba1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:35.334133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 286/558] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 06:30:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:35.335508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534084a-5d2d-4b2d-befa-87c250e419c0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.335795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534084a-5d2d-4b2d-befa-87c250e419c0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.336063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534084a-5d2d-4b2d-befa-87c250e419c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.355798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a534084a-5d2d-4b2d-befa-87c250e419c0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:35.357006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a534084a-5d2d-4b2d-befa-87c250e419c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:35.441623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 287/559] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:35.442435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534084a-5d2d-4b2d-befa-87c250e419c0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:35.486229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585aab9-4238-4596-a042-c3e07677f6c9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:35.527629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:35.545315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:35.546367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.546849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.547413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.555309 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:35.557163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:35.570336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:35.571857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:35.572796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:35.580234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 273/560] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:30:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:35.581360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:35.663595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:35.694293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534084a-5d2d-4b2d-befa-87c250e419c0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:35.736331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e15f-5763-48fd-9211-9806a7714aeb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:35.737269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:35.737890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.738167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.738440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.774116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:35.775120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.775688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.776206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.787552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:35.793291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:35.794302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:35.795132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:35.797134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 274/561] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:35.798057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:35.809426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:35.811267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:35.859614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:35.915543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ac9e-8f97-4737-9bc6-852362642853 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:35.926892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 288/562] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:35.928041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:35.940771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:35.966607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:35.967576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:35.967978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:35.968353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:35.969184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:35.969584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:35.972079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:35.995957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 275/563] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:30:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:35.996994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.043807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79882-fe07-46df-aa31-86b5dd923052 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.088172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.089059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.089601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.090111 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.098124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:36.099964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4555af-f764-4737-a115-5d4ca574c471 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.103066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:36.108294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:36.109487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:36.110542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:36.112718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 289/564] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:36.113637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.142550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.143289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.143633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.143976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.144641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:36.144945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.146330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.154696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 276/565] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:36.155866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.198185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:36.242191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c40be-b849-4e36-bf53-cd79f7aa20ef None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.276561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0d9ab-c739-4273-89f1-f4f47d4f32ef None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.278192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.278906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.279316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.279652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.290163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.291682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:36.307135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:36.308274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:36.309374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:36.309374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.309374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:36.309374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 290/566] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:36.309374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.317712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7e25a-2675-416d-af39-165e03234dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.318354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7e25a-2675-416d-af39-165e03234dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.318628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7e25a-2675-416d-af39-165e03234dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.318902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7e25a-2675-416d-af39-165e03234dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.329641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7e25a-2675-416d-af39-165e03234dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.331080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c7e25a-2675-416d-af39-165e03234dc1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-85c7e25a-2675-416d-af39-165e03234dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:36.344859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:36.345815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:36.346714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:36.346714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.346714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:36.346714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 277/567] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:36.347507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279300c0-5d0d-4cf3-acee-e70202081f7c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.347856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279300c0-5d0d-4cf3-acee-e70202081f7c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.348065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279300c0-5d0d-4cf3-acee-e70202081f7c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.349291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 278/568] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:36.349975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279300c0-5d0d-4cf3-acee-e70202081f7c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.367606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:36.423705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864b2d-1841-4257-aded-742ec736bde5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.444962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:36.460046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279300c0-5d0d-4cf3-acee-e70202081f7c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.462961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.464369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.464654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.464926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.465607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:36.465922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.467338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.475900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 291/569] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:36.476635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.505193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.505828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.506104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.506489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.517693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.519137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:36.534377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:36.535500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:36.541229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:36.541229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.541229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:36.541229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 279/570] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:36.541229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.593603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419854d0-da1f-4b4b-a7b3-4f644f685311 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.607507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:36.630308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9f480-3b96-4fe8-8e87-3145d0d05d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.630935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9f480-3b96-4fe8-8e87-3145d0d05d13 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.631241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9f480-3b96-4fe8-8e87-3145d0d05d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.631516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9f480-3b96-4fe8-8e87-3145d0d05d13 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.632157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9f480-3b96-4fe8-8e87-3145d0d05d13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:36.632470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9f480-3b96-4fe8-8e87-3145d0d05d13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.633840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9f480-3b96-4fe8-8e87-3145d0d05d13 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.645536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fdebc-6647-49bc-a563-f01c78fe305f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.646336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 292/571] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:36.648485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b964ee9-a0bd-4a6b-ba25-d3dfbfb605eb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.648485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b964ee9-a0bd-4a6b-ba25-d3dfbfb605eb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.648485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b964ee9-a0bd-4a6b-ba25-d3dfbfb605eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.666146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4b964ee9-a0bd-4a6b-ba25-d3dfbfb605eb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:36.667378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4b964ee9-a0bd-4a6b-ba25-d3dfbfb605eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:36.686722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.687507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.687898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.688284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.689358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:36.689730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.691167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.703001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:36.707050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 280/572] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:36.708042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.744288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 293/573] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:36.745150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b964ee9-a0bd-4a6b-ba25-d3dfbfb605eb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.874691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b964ee9-a0bd-4a6b-ba25-d3dfbfb605eb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.876466 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:36.880533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd0218-3c6b-41f3-a790-adbd6ca90ab4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:36.916796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.917500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.917845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.918197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.918907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:36.919325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.920803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.937224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 294/574] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:36.937224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:36.953564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:36.954315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:36.954666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:36.955032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:36.968392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:36.969999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:36.985618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:36.986795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:36.987943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:36.987943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:36.987943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:36.992948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 281/575] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:36.993929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.047957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.060213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935fec4-471a-4cc8-9826-3662ea68bd2c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.095662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.095847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb985-5eb3-462a-8822-a7657acc2b4d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.096305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.096589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.096865 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.103576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.105041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:37.116353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:37.117399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:37.118471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 295/576] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:37.119060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.133511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.134155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.134439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.134717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.141652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.143097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.143797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:37.154588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:37.155905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:37.156973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:37.160739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 282/577] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:37.161597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.220930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff6ca5-702e-4803-80bd-50d93855f2dd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.255610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.256246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.256532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.256815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.264383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.265912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.269655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f6dd5-ddd3-4957-ae14-791a273674af None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:37.276969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:37.277964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:37.278930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 296/578] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:37.279547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.287404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.310640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.311186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.311539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.311822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.312483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:37.312787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.314530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.326145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 283/579] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:37.327023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.368905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.387977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754aab9-03dc-4557-8539-8a0ce9c5f7e8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.421135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.421819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.422057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.422345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.427033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15520eb-adb3-4d1c-8207-f719424bcd71 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.433097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:37.438214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:37.439683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 297/580] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:37.440790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.467528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.468179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.468457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.468730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.469427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:37.469732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.471163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.483227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 284/581] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:37.484277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.516849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.543326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52509-2b9a-44ed-80a8-2f369f39e9db None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.580039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.580578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.580862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.581137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.581826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:37.582139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.583631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.595701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 298/582] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:37.596567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.599446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.604028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce9e03-4df5-4505-a08a-08af23862178 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.643565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.644190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.644476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.644755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.645505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:37.645821 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.647296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.668600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 285/583] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:37.669449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.701072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8886b-774a-4494-8d54-c5214fb6a8d7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.735366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.736005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.736287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.736563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.744165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.745613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:37.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:37.758828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 299/584] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:37.764260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.768304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.777702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e8b90-546e-4154-b43c-a724c892deb4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.811748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.812568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.812851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.813124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.813798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:37.814468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.815874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.824017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 286/585] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:37.824767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.828842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:37.870937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea8502-9cfa-469f-a603-f48e2b4623de None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.905652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.906487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.906771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.907048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.907821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f7ca3818552143069b0bacbb5bec7b62) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:37.908138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:37.909593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:37.915264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 300/586] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/role_assignments?user.id=f7ca3818552143069b0bacbb5bec7b62&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:37.915860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:37.935900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6f625-040c-4564-96f1-6c0ef8ae1e36 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:37.977720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:37.978440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:37.978721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:37.979000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/8e1ec869263246f58f5a32d855a3d302/roles/72fda378772c45f4b41c5f973cc7cda2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:37.999728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=8e1ec869263246f58f5a32d855a3d302, role_id=72fda378772c45f4b41c5f973cc7cda2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.001234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.028875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 287/587] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:30:37 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/8e1ec869263246f58f5a32d855a3d302/roles/72fda378772c45f4b41c5f973cc7cda2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:38.028875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.032445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.042625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7488a0-3902-4655-804b-1e9c5db3d4de None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.082760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.095528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.096103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.098238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.098238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.114258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.116713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:38.136601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:38.137812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:38.138934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:38.138934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:38.138934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:38.143350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 301/588] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:38.144640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.165350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a232d-a705-4996-8fd0-ddb9f8d5e864 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.213647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.214289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.214577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.219784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.220093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:38.220956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.221902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.249334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 288/589] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:38.250212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.287870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e2dac-86db-4b0f-a9e3-deb811f10e15 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.326470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.332349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.333002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.333294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.333725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.345766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.347112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:38.362818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:38.364173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:38.365077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:38.365077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:38.365077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:38.365077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 302/590] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:38.365317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.381913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a94bab-552c-4504-ade2-4dbbf245a50f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.402302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.418575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.419592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.420057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.420559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.421941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:38.422581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.424722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.439987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 289/591] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:38.441255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.484698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a4b60-776f-44c0-b2c1-b13c6df9ef76 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.525536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.526809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.527320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.528036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.545657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.547947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:38.569132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:38.570182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:38.571095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:38.571095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:38.571095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:38.571881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d14dbc-e469-4ac5-b6e0-94890ebbad91 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.575543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 303/592] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:38.576620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.597453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.614136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.615018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.615018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.615394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.616054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:38.616467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.618131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.635288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 290/593] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:38.635288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.689488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.715098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385665a-c1d1-4bed-ace9-838484468939 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.744767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07773790-d436-48b3-be1f-f851eae2fc87 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.751032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.751915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.752346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.752767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.760631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.762490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:38.775638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:38.776606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:38.777510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:38.779930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 304/594] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:38.780969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.793604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.794278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.794569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.794843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.795544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:38.795876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.797371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.814677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 291/595] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:38.815609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.861165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.891813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f893d-609f-4bff-83c2-e917af76a9f0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.926986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175da8b0-02c7-47e7-908c-f38f116cd1c0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:38.930048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.930903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.931379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.931799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.942881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.945040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:38.960544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:38.961741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:30:38.962916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:38.969520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:38.969520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:38.969520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:38.969520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:38.969901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:38.969901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:38.969901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:38.970437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 305/596] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:38.971477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:38.987559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:38.992762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 292/597] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:38.993763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.081271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970453-3ffb-4612-a4b6-677a0b44de4c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.103619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:39.107817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa834051-89c4-4767-aeb8-9dfaa024d6a4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.119173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.120120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.120552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.120977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.128546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.130192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:39.142144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:39.143070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:39.144112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:39.146677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 306/598] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:39.146783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.147754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.152066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.152192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.152573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.153398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:39.153849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.156321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.179974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 293/599] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 24 06:30:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:39.179974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.227565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:39.271506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bfbe-c885-4aa9-8f88-a4d2cbb2ab31 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.289217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df34ec-e522-4875-92c7-ffeabe873b93 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.309872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.310894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.311947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.312436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/380786eb9b7848d59f3bb0007726dd2a/roles/72fda378772c45f4b41c5f973cc7cda2` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.328961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.329321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=380786eb9b7848d59f3bb0007726dd2a, role_id=72fda378772c45f4b41c5f973cc7cda2)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.329570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.329851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.330122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.330816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:39.331119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.332106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.332554 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.348633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 307/600] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:30:39 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/380786eb9b7848d59f3bb0007726dd2a/roles/72fda378772c45f4b41c5f973cc7cda2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:39.350024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.355626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 294/601] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:39.370731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.419565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:39.470492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414fe86-1020-4822-9e55-ae312a53b7f5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.486299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:39.513638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.514613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.515049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.515545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.527229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.528970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:39.547588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:39.548730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:39.549622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:39.549622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:39.549622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:39.555547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 308/602] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:39.557637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.622088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc02a11-84b2-490c-a5c5-1102fcb80693 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.659024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.659738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.660014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.660285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/c216522efef4447caac245392d95b985/roles/72fda378772c45f4b41c5f973cc7cda2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.674561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d3ebd-11fa-42b1-abd3-b0e7a3011e12 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.676961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=c216522efef4447caac245392d95b985, role_id=72fda378772c45f4b41c5f973cc7cda2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.678416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.696239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 295/603] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:30:39 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/c216522efef4447caac245392d95b985/roles/72fda378772c45f4b41c5f973cc7cda2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:30:39.716838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aedd5-663d-4ee5-9fc0-731e3617eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.718019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aedd5-663d-4ee5-9fc0-731e3617eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.718515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aedd5-663d-4ee5-9fc0-731e3617eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.719000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aedd5-663d-4ee5-9fc0-731e3617eeb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.720087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aedd5-663d-4ee5-9fc0-731e3617eeb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:39.720743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aedd5-663d-4ee5-9fc0-731e3617eeb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.722602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aedd5-663d-4ee5-9fc0-731e3617eeb7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.738414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 309/604] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:39.744568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.866400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:39.916146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8158321-9994-4545-9229-878a9ba1724b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:39.931854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc4cb-adb1-4be1-9065-7e0423f941cc None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.932476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc4cb-adb1-4be1-9065-7e0423f941cc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.932957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc4cb-adb1-4be1-9065-7e0423f941cc None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.934764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 310/605] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:39.942924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6e9f9-0561-4c28-970d-8ee8696b2355 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.943526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6e9f9-0561-4c28-970d-8ee8696b2355 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.944021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6e9f9-0561-4c28-970d-8ee8696b2355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.964028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:39.964362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:39.964592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:39.964867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:39.970670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-37a6e9f9-0561-4c28-970d-8ee8696b2355 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:39.972525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:39.972775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-37a6e9f9-0561-4c28-970d-8ee8696b2355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:39.973941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:39.989607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:39.990602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:39.991587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:39.995400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 296/606] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:39.996939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:39.998870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:40.104855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 311/607] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:40.142013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85a728-125d-43bc-ab73-8997eed89a6f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:40.142297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85a728-125d-43bc-ab73-8997eed89a6f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:40.142570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85a728-125d-43bc-ab73-8997eed89a6f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:40.144513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 312/608] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:30:40.149791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c06bdd-03db-4618-969a-3b7fae88ffe7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:40.150249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c06bdd-03db-4618-969a-3b7fae88ffe7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:40.150709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c06bdd-03db-4618-969a-3b7fae88ffe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:40.163124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9592549-01a4-49cb-920a-4639c49b049b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:40.183362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-63c06bdd-03db-4618-969a-3b7fae88ffe7 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:40.184802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-63c06bdd-03db-4618-969a-3b7fae88ffe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:40.221835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:40.234906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:40.238534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:40.238931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:40.239293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:40.241043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:40.241422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:40.242936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:40.262020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 297/609] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:40.272018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:40.352574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 313/610] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:40.405033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24808b12-e695-429a-a1d6-cc546087bf23 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:40.411627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c06bdd-03db-4618-969a-3b7fae88ffe7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:40.442854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:40.447240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:40.447240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:40.447240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/2fc4592ef54b41059fa7767a5ca3aaa3/roles/72fda378772c45f4b41c5f973cc7cda2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:40.459213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=2fc4592ef54b41059fa7767a5ca3aaa3, role_id=72fda378772c45f4b41c5f973cc7cda2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:40.460806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:40.478531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 298/611] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:30:40 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/2fc4592ef54b41059fa7767a5ca3aaa3/roles/72fda378772c45f4b41c5f973cc7cda2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:40.596968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:40.614332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c06bdd-03db-4618-969a-3b7fae88ffe7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:40.665400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:40.667167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:40.667167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:40.667167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:40.681801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:40.687175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:40.688150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:40.689388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 314/612] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:40.698582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:40.713386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:40.744382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:40.809478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b097b3-bf86-41dc-af8d-432ff36500a6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:40.831351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82ccb-bb27-468a-ab48-88826a1f1d36 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:40.868271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:40.868840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:40.869195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:40.869486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:40.875927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:40.876787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:40.877227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:40.877653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:40.878488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:40.878952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:40.880530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:40.881341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:40.893229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:40.894289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:40.896285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 315/613] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:40.904279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 299/614] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:40.909791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:40.910516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:40.915529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:41.031631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:41.039620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46452fc2-2072-4684-ab83-c432f1f3ba01 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:41.070760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b8f9f-3526-4863-80a1-44c98adea50b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:41.112169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:41.113047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.113477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.113987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.115140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:41.116471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:41.118232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:41.139211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 300/615] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:41.142734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:41.144052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:41.146520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.146948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.147381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.171232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:41.174926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:41.201470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:41.202794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:41.203851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:41.203851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:41.203851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:41.203851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 316/616] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:41.209222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:41.278879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a07f6-0b2f-4718-9f5b-4ec296c20b32 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:41.323042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:41.324865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.326427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.326846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.350985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:41.352813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:41.382284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:41.383492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:41.384392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:41.384392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:41.384392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:41.387053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:41.394659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 301/617] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:41.398820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:41.419323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:41.478524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9500efe-0d11-407f-9391-4bf5d176bf84 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:41.554822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:41.555769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.556193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.556607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.557418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:41.558935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:41.560495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:41.582416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245c2ee-dde1-4573-8073-75241787d67b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:41.591943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 317/618] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:41.593143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:41.653952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:41.656561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.659896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.660319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.661151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:41.661596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:41.663114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:41.683120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 302/619] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:41.691332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:41.802256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7f44c-dbf4-4ace-af6c-66ab9b79c5ad None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:41.830446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:41.835646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:41.859677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d89d-2520-4986-a8db-6f85613acfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:41.860599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d89d-2520-4986-a8db-6f85613acfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.860887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d89d-2520-4986-a8db-6f85613acfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.861163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d89d-2520-4986-a8db-6f85613acfb0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.862064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890574dd-5f10-4d1a-9692-39216601a774 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:41.869683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d89d-2520-4986-a8db-6f85613acfb0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:41.871622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d89d-2520-4986-a8db-6f85613acfb0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d9b5d89d-2520-4986-a8db-6f85613acfb0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:41.891219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:41.892402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 318/620] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a018a-52b8-4d3b-85b6-06afa6642137 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.893375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a018a-52b8-4d3b-85b6-06afa6642137 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.894101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a018a-52b8-4d3b-85b6-06afa6642137 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.899348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 319/621] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:41.900795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a018a-52b8-4d3b-85b6-06afa6642137 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:41.911285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95410-70ad-4230-be00-140efcb87f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:41.911285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95410-70ad-4230-be00-140efcb87f4d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.911661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95410-70ad-4230-be00-140efcb87f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.911661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95410-70ad-4230-be00-140efcb87f4d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.919710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95410-70ad-4230-be00-140efcb87f4d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:41.921430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95410-70ad-4230-be00-140efcb87f4d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-74c95410-70ad-4230-be00-140efcb87f4d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:41.933847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:41.935723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:41.937781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:41.947217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 303/622] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:30:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:41.947217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2a8f-5472-48d7-928d-1bc00bce8023 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:41.947217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2a8f-5472-48d7-928d-1bc00bce8023 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:41.947217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2a8f-5472-48d7-928d-1bc00bce8023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:41.963005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e4fc2a8f-5472-48d7-928d-1bc00bce8023 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:41.964298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e4fc2a8f-5472-48d7-928d-1bc00bce8023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:42.032702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a018a-52b8-4d3b-85b6-06afa6642137 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.044887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 304/623] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:42.046064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc2a8f-5472-48d7-928d-1bc00bce8023 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.082640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe03389-a7ca-4e3c-b357-88175c3f5089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.083449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe03389-a7ca-4e3c-b357-88175c3f5089 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.084601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe03389-a7ca-4e3c-b357-88175c3f5089 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.085694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe03389-a7ca-4e3c-b357-88175c3f5089 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.086447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe03389-a7ca-4e3c-b357-88175c3f5089 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:42.086761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe03389-a7ca-4e3c-b357-88175c3f5089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:42.088278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe03389-a7ca-4e3c-b357-88175c3f5089 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:42.126294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 320/624] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 06:30:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:42.127733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473184a0-e7bf-49f6-8374-790a872ff1f6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.128027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473184a0-e7bf-49f6-8374-790a872ff1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.128309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473184a0-e7bf-49f6-8374-790a872ff1f6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.130299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 321/625] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:42.131115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473184a0-e7bf-49f6-8374-790a872ff1f6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.154097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.173510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.199711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc2a8f-5472-48d7-928d-1bc00bce8023 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.243228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa00beb-b491-4785-84f9-633d78cbb498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.243228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa00beb-b491-4785-84f9-633d78cbb498 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.243228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa00beb-b491-4785-84f9-633d78cbb498 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.243228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa00beb-b491-4785-84f9-633d78cbb498 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.243650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa00beb-b491-4785-84f9-633d78cbb498 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:42.243650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa00beb-b491-4785-84f9-633d78cbb498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:42.244407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa00beb-b491-4785-84f9-633d78cbb498 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:42.247891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473184a0-e7bf-49f6-8374-790a872ff1f6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.263986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 305/626] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:42.265528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f52e16-0b1d-4096-bb07-17dd98186a62 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.265952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f52e16-0b1d-4096-bb07-17dd98186a62 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.266389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f52e16-0b1d-4096-bb07-17dd98186a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.285455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-32f52e16-0b1d-4096-bb07-17dd98186a62 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:42.286909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-32f52e16-0b1d-4096-bb07-17dd98186a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:42.288777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.289428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.289711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.289994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.300698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:42.302234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:42.318387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:42.319644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:42.320593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:42.320593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:42.320593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:42.324126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 322/627] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:42.325937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.368990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 306/628] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:42.369917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f52e16-0b1d-4096-bb07-17dd98186a62 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.418276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.441730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8833f-12f6-41b0-85a6-4c521d7bef59 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.458287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.490970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.491627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.491922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.492201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.509466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f52e16-0b1d-4096-bb07-17dd98186a62 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.560927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.562202 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.562513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.562793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.563478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:42.563785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:42.565553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:42.577958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 307/629] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:42.578858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.586898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:42.588692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:42.683784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141cbb-83f6-456d-b3cc-acf2aaf93fb3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.685567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.698975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.703564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 323/630] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:42.704255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.724835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422292a-788c-4f0d-b341-26851867617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.725483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422292a-788c-4f0d-b341-26851867617f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.725768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422292a-788c-4f0d-b341-26851867617f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.726042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422292a-788c-4f0d-b341-26851867617f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.736361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422292a-788c-4f0d-b341-26851867617f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b422292a-788c-4f0d-b341-26851867617f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:42.741552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:42.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 308/631] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:42.743485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422292a-788c-4f0d-b341-26851867617f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.811648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f4890-6e5f-4e58-902d-53ea522e988a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.845690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.846322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.846597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.846867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.850720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422292a-788c-4f0d-b341-26851867617f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:42.853363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:42.854822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:42.865699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:42.866702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:42.867675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 324/632] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:42.869679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.893692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef83ebf-3bf4-4cd6-94da-914c2840a666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:42.894335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef83ebf-3bf4-4cd6-94da-914c2840a666 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.894618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef83ebf-3bf4-4cd6-94da-914c2840a666 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.894891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef83ebf-3bf4-4cd6-94da-914c2840a666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.905360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef83ebf-3bf4-4cd6-94da-914c2840a666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4ef83ebf-3bf4-4cd6-94da-914c2840a666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:42.910571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:42.911731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:42.912611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 309/633] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:42.913340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c5ef5-a425-4407-9ad6-c11a577a8472 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:42.913340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c5ef5-a425-4407-9ad6-c11a577a8472 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:42.913465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c5ef5-a425-4407-9ad6-c11a577a8472 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:42.914784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 310/634] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:42.919654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c5ef5-a425-4407-9ad6-c11a577a8472 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:42.921161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.951945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:42.978026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f648fa-1106-4477-8dab-42a13725b3d6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.013611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.014271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.014591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.015024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.025030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.026787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.041082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c5ef5-a425-4407-9ad6-c11a577a8472 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:43.042761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:43.048779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:43.049824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:43.049824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:43.049824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:43.049824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 325/635] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:43.049824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.074456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.075072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.075378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.075652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.076317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:43.076617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.078079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.090439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 311/636] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:43.090439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.148736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.152879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93480bc3-0ae4-4553-be80-8aa4e1bf976f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.187947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.188645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff58aa-2b1b-4c81-a35f-cc8e6298c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.189700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff58aa-2b1b-4c81-a35f-cc8e6298c95e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.190747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff58aa-2b1b-4c81-a35f-cc8e6298c95e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.191035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff58aa-2b1b-4c81-a35f-cc8e6298c95e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.191773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff58aa-2b1b-4c81-a35f-cc8e6298c95e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:43.192097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff58aa-2b1b-4c81-a35f-cc8e6298c95e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.193559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff58aa-2b1b-4c81-a35f-cc8e6298c95e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.199643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f39a3-8420-4b1e-851c-1807d06b7a76 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.206117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 326/637] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:43.207423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e933f-3b30-47b9-936d-bd7536f08e71 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.207778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e933f-3b30-47b9-936d-bd7536f08e71 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.208122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e933f-3b30-47b9-936d-bd7536f08e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.226069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-115e933f-3b30-47b9-936d-bd7536f08e71 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:43.227287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-115e933f-3b30-47b9-936d-bd7536f08e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:43.238518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.239243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.239533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.239809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.240481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:43.240784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.242153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.250349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 312/638] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:43.251493 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.299873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 327/639] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:43.300876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e933f-3b30-47b9-936d-bd7536f08e71 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.358154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ac1d-4f7e-4351-b7a2-5c1939b1cdac None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.399473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.400968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.401711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.402096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.402435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.403179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:43.403584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.405082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.414693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e933f-3b30-47b9-936d-bd7536f08e71 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.421818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 313/640] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:43.423525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.443813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.454107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.454908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.455405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.455642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.466152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.467674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:43.482219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:43.483228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:43.484819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:43.484819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:43.484819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:43.487331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 328/641] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:43.489353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.538696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62a9d3-7530-4ffc-bcca-e2cc48759930 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.573881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.574545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.574827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.575101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f7ca3818552143069b0bacbb5bec7b62/roles/72fda378772c45f4b41c5f973cc7cda2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.590755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=f7ca3818552143069b0bacbb5bec7b62, role_id=72fda378772c45f4b41c5f973cc7cda2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.592248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.604250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19caa3e-8885-4cb8-9217-4b0a0c141bf3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.605852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 314/642] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:30:43 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f7ca3818552143069b0bacbb5bec7b62/roles/72fda378772c45f4b41c5f973cc7cda2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:43.607097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.653213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.655072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.655835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.655835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.666295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.668600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.670301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:43.686850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:43.688274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:43.689717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:43.689717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:43.689717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:43.689717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 329/643] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:43.690226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.699570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.744159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8132e8-01c2-4d13-ac83-6180a890eba7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.783935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.784709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.785064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.785409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.793753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.795341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:43.807757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:43.808895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:43.810070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:43.811269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 315/644] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:43.813243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.815401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06030c-6b55-49fa-965d-7568ba8ecf0d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.851666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.852385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.852678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.852956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.853616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:43.853926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:43.855386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:43.868555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 330/645] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:43.869220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:43.924899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.929959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854c17c-af78-4914-9df7-85875e6a74b0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.960011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:43.969548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:43.970307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:43.970688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:43.971061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:43.979165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8b113-ec3d-4a12-8d23-cdd93d1268a6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:43.982256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:43.987466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:43.988626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:43.989721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:43.990643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 316/646] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:43.995325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.020068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.020689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.020970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.021267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.021899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:44.022209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.023630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.035670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 331/647] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:44.036499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.105170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63191899-26ff-49bf-8daf-8e230f6cc706 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.143685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.144307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.144592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.144867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.145572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:44.145880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.147347 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.152303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e8c00-9f5e-4f87-9bac-2a3198e4790a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.160650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 317/648] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:44.161360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.172774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:44.197319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.198101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.198481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.198852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.207587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.209199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.218405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:44.222268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:44.223451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:30:44.224665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:44.226079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 332/649] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:44.226843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.283185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3b6b5-3f92-4216-afc0-9439236d672a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.319590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.320200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.320500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.320776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.321454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:44.321761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.323170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.331464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 318/650] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:44.332181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.338532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db026e7b-f641-4e78-aeb3-621dd9b2c25f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.375230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.375230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.375230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.375568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.386829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.388474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:44.399870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:44.401036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:30:44.402209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:44.406536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 333/651] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:44.407518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.441585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:44.447061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6b741-015a-4bdf-b69c-16c265100dac None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.456862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:44.485971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.486693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.487009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.487349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.489089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=8e1ec869263246f58f5a32d855a3d302) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:44.489416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.490861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.498081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 319/652] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/role_assignments?user.id=8e1ec869263246f58f5a32d855a3d302&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:44.499374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.522895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106030ea-eabb-4dc6-a69b-4914fa4b5893 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.562904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.563861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.564657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.565132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.565935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:44.566255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.567820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.593299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 334/653] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:44.593299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.661596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee31159-b783-4f02-8d90-ff9c4480c92e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.710331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:44.716916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.716916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.717239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.717438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.726286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e88987-ed57-4a0d-951e-ab9f577a7801 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.732017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.734005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:44.751122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:44.752211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:44.753171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:44.753171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:44.753171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:44.753171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 320/654] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:44.753171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.776678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.777304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.777594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.777870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.778980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:44.778980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.780432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.784242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:44.802282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 335/655] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:44.802282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.881820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304790bf-06b5-4503-8f3f-cc7199f95011 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.916385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.917365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.917951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.918455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.925637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb592-7dac-4e6d-a009-f41479e84619 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:44.968281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:44.969320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:44.969609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:44.969885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:44.970634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:44.970840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:44.972258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:44.985133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 336/656] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:44.986020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:44.997345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:44.999255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.001469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.055552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:45.095314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63ccee-0d53-48b0-92d4-79b1748aed20 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:45.136305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 321/657] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:45.137540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:45.152211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:45.153204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.153693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.154219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.251789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f47b-e51e-4653-8ce0-face9c6fb553 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:45.261187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.262902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.276726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:45.286306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:45.297580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:45.298575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.299118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.299497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.307419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.309128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:45.321749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:45.322913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:30:45.323923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:45.328759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 322/658] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:45.329699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:45.402024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 337/659] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:45.402024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:45.459085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534127a-064b-4d56-862f-a2f78309c076 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:45.495139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:45.496154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.496620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.497098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.508479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.510215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.517402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c9f4-0c1e-4492-a84b-a2cbbb2ac1ab None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:45.525583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:45.526615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:45.527563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:45.527563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:45.527563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:45.527737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:45.529733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 323/660] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:45.531329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:45.558325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeac2e6-540f-4c80-8e56-fe5fb62d15c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:45.558942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeac2e6-540f-4c80-8e56-fe5fb62d15c4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.559249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeac2e6-540f-4c80-8e56-fe5fb62d15c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.559527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeac2e6-540f-4c80-8e56-fe5fb62d15c4 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.560353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeac2e6-540f-4c80-8e56-fe5fb62d15c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:45.560718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeac2e6-540f-4c80-8e56-fe5fb62d15c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.562351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeac2e6-540f-4c80-8e56-fe5fb62d15c4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.570041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:45.585068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 338/661] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Aug 24 06:30:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:45.586162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f05a-c97f-4daf-896d-ee0376b394af None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.586453 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f05a-c97f-4daf-896d-ee0376b394af None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.586730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f05a-c97f-4daf-896d-ee0376b394af None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.592197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 339/662] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:45.592197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101f05a-c97f-4daf-896d-ee0376b394af None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:45.648776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0469c2b-7292-45e8-a93e-cdb011c2306f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:45.692330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:45.693017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.693370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.693715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.706127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.707664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.708960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101f05a-c97f-4daf-896d-ee0376b394af None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:45.722422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:45.723731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:45.725006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:45.725006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:45.725006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:45.725006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 324/663] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:45.725792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:45.751617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08103ed-cff8-4b0c-a4ae-afb3aef3e3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:45.752345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08103ed-cff8-4b0c-a4ae-afb3aef3e3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.752701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08103ed-cff8-4b0c-a4ae-afb3aef3e3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.753046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08103ed-cff8-4b0c-a4ae-afb3aef3e3e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.753770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08103ed-cff8-4b0c-a4ae-afb3aef3e3e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:45.754145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08103ed-cff8-4b0c-a4ae-afb3aef3e3e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.755736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08103ed-cff8-4b0c-a4ae-afb3aef3e3e1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.767521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 340/664] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:45.769033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ce25-8ced-4a20-9c4c-c00ef1fa5f48 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.769329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ce25-8ced-4a20-9c4c-c00ef1fa5f48 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.769611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ce25-8ced-4a20-9c4c-c00ef1fa5f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.789207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1934ce25-8ced-4a20-9c4c-c00ef1fa5f48 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:45.790439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1934ce25-8ced-4a20-9c4c-c00ef1fa5f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:45.813701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:45.831289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:45.845635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d967cdd-c5ac-42aa-9289-8218109783a0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:45.872151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 341/665] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:45.872825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934ce25-8ced-4a20-9c4c-c00ef1fa5f48 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:45.892719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:45.893749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:45.894772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:45.895299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:45.983086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:45.984573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:45.991300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934ce25-8ced-4a20-9c4c-c00ef1fa5f48 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.037647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.038846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.039344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.039938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.041067 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:46.041575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.043141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.057222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 342/666] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:46.058151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.078981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.092350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.110355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 325/667] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:46.111620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.177601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b17c2-fc80-4001-a346-a6e68b78279c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.226251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.227718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.228168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.228932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.238237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.239665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.240704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39c535-2c10-4228-88ff-63d8d970cc97 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:46.251414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:46.252363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 343/668] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:46.253270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.283753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.284397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.284665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.284942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.292108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.293546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:46.305119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:46.306103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 326/669] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:46.307046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.347392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.366958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.375936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c08f3-bdc5-404d-844c-f424e1a81594 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.414465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.415112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.415449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.415729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.419674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9c85d-15e6-4e9d-a5ff-7b6d0f805920 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.426375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.427870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:46.446161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:46.447218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:46.448157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:46.448157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:46.448157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:46.451334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 344/670] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:46.452443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.458490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.459137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.459459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.459738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.470491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:46.475291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:46.476254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:46.477117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 327/671] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:46.477944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.574898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031075f-ac24-49a2-bcf9-f665af83dce1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.590348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96fce1-7502-4f77-917e-24d379f57e19 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.601412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.601989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.612570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.613195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.613477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.613830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.614536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:46.614847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.616344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.630121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.630694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 345/672] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:46.630802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.631001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.631306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.631549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.632035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:46.632312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.633742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.645681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 328/673] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:46.646351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.735166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be67b8-2a2d-44f1-9ec8-ce968aa6386e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.748881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43005eb-ecce-428d-9e54-99362f78090c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.770791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ee53f-aebb-4cb1-bfe9-e230d9ccfb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.771452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ee53f-aebb-4cb1-bfe9-e230d9ccfb53 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.771758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ee53f-aebb-4cb1-bfe9-e230d9ccfb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.772039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ee53f-aebb-4cb1-bfe9-e230d9ccfb53 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.772701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ee53f-aebb-4cb1-bfe9-e230d9ccfb53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:46.773006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ee53f-aebb-4cb1-bfe9-e230d9ccfb53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.774417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ee53f-aebb-4cb1-bfe9-e230d9ccfb53 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.785241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.785816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 346/674] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:46.785936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.786113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.786390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.786996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54e249-1ef6-4e1b-bec8-fa86604b19dd None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.787166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:46.787350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54e249-1ef6-4e1b-bec8-fa86604b19dd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.787454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.787615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54e249-1ef6-4e1b-bec8-fa86604b19dd None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.788936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 347/675] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:46.789037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.789469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54e249-1ef6-4e1b-bec8-fa86604b19dd None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.801714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 329/676] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:46.802559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.834575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.847688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:46.909112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54e249-1ef6-4e1b-bec8-fa86604b19dd None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.915674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe29854-3ad5-4c27-8015-caf521b4ea31 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:46.944037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.944893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.945176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.945449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.946173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=380786eb9b7848d59f3bb0007726dd2a) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:46.946484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.948090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.949789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd156c5-33eb-42d8-b986-47728583d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:46.950458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd156c5-33eb-42d8-b986-47728583d292 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.950744 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd156c5-33eb-42d8-b986-47728583d292 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.951044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd156c5-33eb-42d8-b986-47728583d292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.951806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd156c5-33eb-42d8-b986-47728583d292 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=c216522efef4447caac245392d95b985) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:46.952251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd156c5-33eb-42d8-b986-47728583d292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:46.953720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd156c5-33eb-42d8-b986-47728583d292 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:46.954004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 348/677] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/role_assignments?user.id=380786eb9b7848d59f3bb0007726dd2a&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:46.954778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:46.965739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 330/678] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/role_assignments?user.id=c216522efef4447caac245392d95b985&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:46.967113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921ab41-afdd-46ba-a5fc-3edf1924e934 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:46.967450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921ab41-afdd-46ba-a5fc-3edf1924e934 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:46.967790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921ab41-afdd-46ba-a5fc-3edf1924e934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:46.996267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d921ab41-afdd-46ba-a5fc-3edf1924e934 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:46.997946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d921ab41-afdd-46ba-a5fc-3edf1924e934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:47.087981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 331/679] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:47.089355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921ab41-afdd-46ba-a5fc-3edf1924e934 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.114807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea098be-3d6b-4ece-b1b3-3b3358a2d5e6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.136224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:47.137414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:47.168657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:47.169698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:47.170700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:47.171332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:47.179875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:47.180902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:47.194642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:47.195666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:47.196625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:47.199316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 349/680] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:47.202242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.234974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921ab41-afdd-46ba-a5fc-3edf1924e934 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.281377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:47.283156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:47.284366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:47.284645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:47.298368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:47.300197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:47.317983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:47.319007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:47.319957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:47.319957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:47.319957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:47.326656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 332/681] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:47.328037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.337687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cec18-972c-4e08-8444-bfea8747719e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.375863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:47.376936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:47.377436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:47.377943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:47.378851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:47.379372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:47.381196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:47.395468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 350/682] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:47.416570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.418012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:47.444159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e1260-2756-48b2-acd1-3c1ea3acc361 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.459775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:47.480337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:47.480962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:47.481250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:47.481526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:47.482334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:47.482534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:47.484115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:47.500998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 333/683] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:47.501944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.537076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14055526-c621-41ea-b4db-5cd4d35a136e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.581556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:47.582563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:47.583063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:47.583563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:47.584489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=2fc4592ef54b41059fa7767a5ca3aaa3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:47.585015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:47.586735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:47.593655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 351/684] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/role_assignments?user.id=2fc4592ef54b41059fa7767a5ca3aaa3&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:47.594790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.616344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295967-3ccf-406e-8749-dad71dfac9f4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.705284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:47.705895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:47.706180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:47.706452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:47.716302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:47.719444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68110ba-2187-4717-bfba-63d972c78771 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.743627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:47.752305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:47.757103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:47.758042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:47.758908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 334/685] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:47.766296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.768720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:47.769395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:47.769687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:47.769977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:47.783288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:47.787324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:47.800706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:47.801786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:47.802752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:47.807303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 352/686] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:47.811553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:47.944890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd06ad-84e8-4a7f-9bc7-01c8ad2d4d0d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:47.960635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703803dc-d6fe-43af-9f5e-04f7b6ee7403 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:48.004238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:48.005275 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:48.005770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:48.006271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:48.007153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:48.007689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:48.009312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:48.027717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:48.028695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:48.029181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:48.029708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:48.030766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:48.031356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:48.033243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:48.038039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 335/687] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:48.043749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:48.059439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 353/688] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 06:30:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:30:48.063551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:48.081862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:48.182949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181ab3f-91a6-4baf-8ad8-ab95a22bc274 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:48.244688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:48.245791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:48.246275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:48.246760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:48.271376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:48.370904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cfae56-8651-48ac-b215-e2e951183d0e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:48.419575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:48.420610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:48.421296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:48.421901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:48.432061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:48.433682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:48.437595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:48.439889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:48.458983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:48.460078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:48.461029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:48.461029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:48.461029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:48.463871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 336/689] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:48.472036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:48.492820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:48.598552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a514c-35d8-4f99-b52d-c64754f3e5ea None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:48.611502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:48.640856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:48.641856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:48.642365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:48.642876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:48.644402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:48.644940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:48.646533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:48.661511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 337/690] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:48.672133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:48.795330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f94f3-d222-4045-bdcf-611dc0401545 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:48.830957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 354/691] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:30:48.835991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:48.846336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:48.847403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:48.848578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:48.849077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:48.867628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:48.871809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:48.910947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:48.912019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:48.913033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 338/692] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:30:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:30:48.917938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:48.988113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:49.092570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b9be3-7c15-4721-965a-e12781db0c10 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:49.126470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:49.156261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:49.157339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.157785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.158219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.177186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:49.179168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:49.184456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3309c6-f75e-40b5-8bc3-0d193ed588c2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:49.208931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:49.210467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:49.211635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:49.211635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:49.211635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:49.218182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 355/693] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:49.225641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:49.271413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a3d87-71f9-4ec5-9198-63ed2e9606eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:49.272646 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a3d87-71f9-4ec5-9198-63ed2e9606eb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.273076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a3d87-71f9-4ec5-9198-63ed2e9606eb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.273530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a3d87-71f9-4ec5-9198-63ed2e9606eb None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.274436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a3d87-71f9-4ec5-9198-63ed2e9606eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:49.274901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a3d87-71f9-4ec5-9198-63ed2e9606eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:49.276603 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a3d87-71f9-4ec5-9198-63ed2e9606eb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:49.307228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 339/694] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:30:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:49.309289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d692863-a18d-4926-a5a2-98872053aa51 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.309999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d692863-a18d-4926-a5a2-98872053aa51 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.310427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d692863-a18d-4926-a5a2-98872053aa51 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.312315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 340/695] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:49.313555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d692863-a18d-4926-a5a2-98872053aa51 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:49.395711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde29ac-02f1-460c-b8c7-ac2d758f7deb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:49.435647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:49.440408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d2814-b58a-4e4f-afca-e0052be20fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:49.441143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2814-b58a-4e4f-afca-e0052be20fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.441516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2814-b58a-4e4f-afca-e0052be20fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.442044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2814-b58a-4e4f-afca-e0052be20fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.442814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d2814-b58a-4e4f-afca-e0052be20fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:49.443542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d2814-b58a-4e4f-afca-e0052be20fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:49.445455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d2814-b58a-4e4f-afca-e0052be20fc4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:49.453791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d692863-a18d-4926-a5a2-98872053aa51 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:49.466252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 356/696] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:49.466797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd50e6-7459-4c84-815a-1b734fa23949 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.467208 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd50e6-7459-4c84-815a-1b734fa23949 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.467587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd50e6-7459-4c84-815a-1b734fa23949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.488216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-facd50e6-7459-4c84-815a-1b734fa23949 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:49.489456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-facd50e6-7459-4c84-815a-1b734fa23949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:49.500114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cac6dd-a991-4586-8bcc-250cab68c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:49.500869 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cac6dd-a991-4586-8bcc-250cab68c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.501374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cac6dd-a991-4586-8bcc-250cab68c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.501585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cac6dd-a991-4586-8bcc-250cab68c5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.511763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:49.568465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 357/697] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:49.569811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816891-64c5-4975-8316-33df367361bf None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.570001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816891-64c5-4975-8316-33df367361bf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.571047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816891-64c5-4975-8316-33df367361bf None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.572508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 358/698] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:49.572732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6816891-64c5-4975-8316-33df367361bf None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:49.583793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cac6dd-a991-4586-8bcc-250cab68c5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:49.586471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cac6dd-a991-4586-8bcc-250cab68c5a0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:49.693265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6816891-64c5-4975-8316-33df367361bf None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:49.704327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:49.719980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 341/699] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:49.721538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d48007-fb71-4e6d-a74f-3269d3107d0c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.722026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d48007-fb71-4e6d-a74f-3269d3107d0c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.722466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d48007-fb71-4e6d-a74f-3269d3107d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.737829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:49.738543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.738887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.739244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.743071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-45d48007-fb71-4e6d-a74f-3269d3107d0c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:49.744559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-45d48007-fb71-4e6d-a74f-3269d3107d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:49.747676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:49.749468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:49.761534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:49.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:49.763995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:49.767693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 359/700] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:49.768480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:49.790643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:49.826306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 342/701] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:49.827649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d48007-fb71-4e6d-a74f-3269d3107d0c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:49.882467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78a0a3-b283-41e4-82f4-14bdbe2087f0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:49.924395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:49.925033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.925311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.925584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.935003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d48007-fb71-4e6d-a74f-3269d3107d0c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:49.937087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:49.938734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:49.953655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:49.954679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:49.955662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:49.955662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:49.955662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:49.955662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 360/702] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:49.957653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:49.960045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:49.977889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:49.978978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:49.979483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:49.979958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:49.980860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:49.981367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:49.983043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:49.995988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 343/703] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:49.997173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.041031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:50.077335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51403ee5-2c59-4380-9df3-7fb7f6a3efeb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.104320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697947f9-2853-40b5-b407-e8d49fcf7d06 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.112414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.113057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.113336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.113609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.126492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.127998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.142398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:50.142722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:50.143844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:50.144766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:50.144766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:50.144766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:50.144948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.144948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.144948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.145127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 361/704] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:50.145204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:50.145204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.145329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.145873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.159122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 344/705] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:50.160357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.200908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:50.251510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4c3c9-dcc3-47d8-9024-3be9b90ce80d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.267344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052672d-1921-4393-8393-7d538bdbb3f7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.270944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:50.285844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fea37-3463-4b6c-9e56-f968c4222abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.286528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fea37-3463-4b6c-9e56-f968c4222abb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.286807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fea37-3463-4b6c-9e56-f968c4222abb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.287075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fea37-3463-4b6c-9e56-f968c4222abb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.287833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fea37-3463-4b6c-9e56-f968c4222abb None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f7ca3818552143069b0bacbb5bec7b62) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:50.288136 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fea37-3463-4b6c-9e56-f968c4222abb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.289569 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fea37-3463-4b6c-9e56-f968c4222abb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.295176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 362/706] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/role_assignments?user.id=f7ca3818552143069b0bacbb5bec7b62&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:50.296361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5c89-e970-40f9-9610-49a00690e4fb None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.296839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5c89-e970-40f9-9610-49a00690e4fb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.297106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5c89-e970-40f9-9610-49a00690e4fb None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.298316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 363/707] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:50.298316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a5c89-e970-40f9-9610-49a00690e4fb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.307092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.307788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.308070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.308339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.308970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.310361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:50.323204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:50.324391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:50.328535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 345/708] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:50.329354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.420405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a5c89-e970-40f9-9610-49a00690e4fb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.445851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e647aa9-18ea-4ca8-b018-30a5f0153376 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.448321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:50.466614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f716a69-e0ad-44de-b21d-6005ef23053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.467334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f716a69-e0ad-44de-b21d-6005ef23053f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.467800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f716a69-e0ad-44de-b21d-6005ef23053f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.468718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f716a69-e0ad-44de-b21d-6005ef23053f None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.469457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f716a69-e0ad-44de-b21d-6005ef23053f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:50.469831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f716a69-e0ad-44de-b21d-6005ef23053f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.471344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f716a69-e0ad-44de-b21d-6005ef23053f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.485312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 364/709] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:50.486862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923b05e-ac14-45f6-b8bb-ee7b5f2e29e5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.487279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923b05e-ac14-45f6-b8bb-ee7b5f2e29e5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.487558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923b05e-ac14-45f6-b8bb-ee7b5f2e29e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.487862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.488501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.488778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.489047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.496463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.497901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.508640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3923b05e-ac14-45f6-b8bb-ee7b5f2e29e5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:50.509850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3923b05e-ac14-45f6-b8bb-ee7b5f2e29e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:50.510835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:50.511798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 346/710] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:50.512731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.559993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:50.591725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 365/711] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:50.591725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923b05e-ac14-45f6-b8bb-ee7b5f2e29e5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.631899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea38fb-bc69-4ec7-ad3a-4c2c1c41b9fb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.677062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.677699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.678393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.678675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.679596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:50.679911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.682543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.688969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 347/712] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:50.690243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.726118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:50.734331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923b05e-ac14-45f6-b8bb-ee7b5f2e29e5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.778833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.779493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.779779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.780057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.781153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.782247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.815581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf1faf-a109-4446-8d5d-736e137c6f0b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:50.820044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 366/713] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:30:50 2022] POST /identity/v3/users => generated 254 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:50.823087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.882019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:50.892874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:50.893563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:50.893818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:50.894791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:50.895554 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:50.899437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:50.902233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:50.926167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 348/714] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:50.926167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:50.984246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752c572-5f78-4814-93f0-7202033f0f8e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:51.030023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:51.031482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:51.031694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:51.031987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:51.032655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:51.032969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:51.034381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:51.043662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 367/715] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:51.045565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:51.053645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:51.072967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f4004-876a-4cf9-b0b4-116e6707bb32 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:51.121729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:51.122439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:51.122872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:51.122979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] PATH_INFO: `/v3/services/network` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:51.124389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:51.125413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:51.137506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:51.138593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:51.139512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 349/716] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:50 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:51.145867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:51.221068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:51.226869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bc6df-ece6-4ba6-9951-f7768644c278 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:51.266658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:51.267336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:51.268914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:51.268914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:51.268914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=8e1ec869263246f58f5a32d855a3d302) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:51.268914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:51.270360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:51.283457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 368/717] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:51 2022] GET /identity/v3/role_assignments?user.id=8e1ec869263246f58f5a32d855a3d302&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:51.381867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a52b3a-ca5a-47f0-a281-9bcfed00ea87 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:51.428020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb9d46-d664-45bc-a00c-df5eec2145fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:51.428914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb9d46-d664-45bc-a00c-df5eec2145fd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:51.429451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb9d46-d664-45bc-a00c-df5eec2145fd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:51.429858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb9d46-d664-45bc-a00c-df5eec2145fd None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:51.430642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb9d46-d664-45bc-a00c-df5eec2145fd None admin] RBAC: Adding query filter params (name=network) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:51.431104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb9d46-d664-45bc-a00c-df5eec2145fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:51.432751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb9d46-d664-45bc-a00c-df5eec2145fd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:51.440187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 350/718] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:30:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:51.448423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:51.571247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e11519-ee43-4143-bf6b-992bede90a48 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:51.625249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03d0a8-198b-4529-9549-1820a984db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:51.626310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03d0a8-198b-4529-9549-1820a984db80 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:51.627252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03d0a8-198b-4529-9549-1820a984db80 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:51.627252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03d0a8-198b-4529-9549-1820a984db80 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:51.628454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03d0a8-198b-4529-9549-1820a984db80 None admin] RBAC: Adding query filter params (type=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:51.628454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03d0a8-198b-4529-9549-1820a984db80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:51.630943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03d0a8-198b-4529-9549-1820a984db80 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:51.639218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 369/719] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:30:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:52.606605 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54136c0-d809-47d6-884b-36bb2b6d9cc1 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:52.607164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54136c0-d809-47d6-884b-36bb2b6d9cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:52.607164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54136c0-d809-47d6-884b-36bb2b6d9cc1 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:52.608763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 351/720] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:52.615747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e13a2-3c18-4a00-a365-344d57de9e1f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:52.615747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e13a2-3c18-4a00-a365-344d57de9e1f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:52.615747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e13a2-3c18-4a00-a365-344d57de9e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:52.638657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9d0e13a2-3c18-4a00-a365-344d57de9e1f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:52.641808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9d0e13a2-3c18-4a00-a365-344d57de9e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:52.662735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:52.718761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 370/721] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:53.105204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54136c0-d809-47d6-884b-36bb2b6d9cc1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:53.122745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:53.225589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54136c0-d809-47d6-884b-36bb2b6d9cc1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:53.308835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7de4cd-136a-4e5e-9d8e-ad818ec9a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:53.309489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7de4cd-136a-4e5e-9d8e-ad818ec9a76b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:53.309769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7de4cd-136a-4e5e-9d8e-ad818ec9a76b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:53.310040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7de4cd-136a-4e5e-9d8e-ad818ec9a76b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:53.337455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7de4cd-136a-4e5e-9d8e-ad818ec9a76b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-af7de4cd-136a-4e5e-9d8e-ad818ec9a76b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:53.343463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:53.344693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:53.345781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:53.351569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 352/722] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:53.357333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e13a2-3c18-4a00-a365-344d57de9e1f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:53.601510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e13a2-3c18-4a00-a365-344d57de9e1f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:53.668540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e2687-66e3-41b9-b143-e0401b4cba2d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:53.668540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e2687-66e3-41b9-b143-e0401b4cba2d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:53.668540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e2687-66e3-41b9-b143-e0401b4cba2d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:53.669688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 353/723] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:53.679326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4608-c155-4df7-9afb-0910acfb28a0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:53.679326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4608-c155-4df7-9afb-0910acfb28a0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:53.679326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4608-c155-4df7-9afb-0910acfb28a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:53.699933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:53.700620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:53.701022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:53.701189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:53.701832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:53.702295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:53.704406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:53.704951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-30bd4608-c155-4df7-9afb-0910acfb28a0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:53.706468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-30bd4608-c155-4df7-9afb-0910acfb28a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:53.729500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 371/724] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:53.740270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:53.792025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 354/725] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:30:53.804471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:53.844843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfd66f-7526-4a29-bb04-a2cb28ebbafa None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:53.845163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfd66f-7526-4a29-bb04-a2cb28ebbafa None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:53.845389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfd66f-7526-4a29-bb04-a2cb28ebbafa None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:53.846763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 355/726] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:53.855009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e496d2-841b-49a2-a248-880ec5c03068 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:53.855382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e496d2-841b-49a2-a248-880ec5c03068 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:53.855607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e496d2-841b-49a2-a248-880ec5c03068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:53.874997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:53.894383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-72e496d2-841b-49a2-a248-880ec5c03068 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:53.898124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-72e496d2-841b-49a2-a248-880ec5c03068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:54.011231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3a64e-0710-4ed9-b9dc-ab9f9add547f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:54.032356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 356/727] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:54.033686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3612b9-67e3-4c77-a847-1a558c6ae30f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.033972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3612b9-67e3-4c77-a847-1a558c6ae30f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.034242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3612b9-67e3-4c77-a847-1a558c6ae30f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.035869 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 357/728] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:54.042026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba689a9-beb8-42b2-a5b9-56ccca9ceef6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.042478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba689a9-beb8-42b2-a5b9-56ccca9ceef6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.042888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba689a9-beb8-42b2-a5b9-56ccca9ceef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.076906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-aba689a9-beb8-42b2-a5b9-56ccca9ceef6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:54.078942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-aba689a9-beb8-42b2-a5b9-56ccca9ceef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:54.091317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:54.091317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.091317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.091317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.111228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:54.111228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:54.131313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:54.132565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:54.133652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:54.133652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:54.133652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:54.133652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 372/729] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:54.134731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:54.213566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:54.217468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 358/730] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:54.218774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba689a9-beb8-42b2-a5b9-56ccca9ceef6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:54.257764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2b7c6-e9c3-419a-8bb2-76837a57692a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:54.311814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:54.312824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.313359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.313890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.315183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:54.319787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:54.321392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:54.359668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 373/731] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:54.366830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:54.368386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:54.427225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba689a9-beb8-42b2-a5b9-56ccca9ceef6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:54.475321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036efc17-36bb-46ba-94c1-370d8e7518d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:54.475321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036efc17-36bb-46ba-94c1-370d8e7518d9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.475321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036efc17-36bb-46ba-94c1-370d8e7518d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.475321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036efc17-36bb-46ba-94c1-370d8e7518d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.488869 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036efc17-36bb-46ba-94c1-370d8e7518d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-036efc17-36bb-46ba-94c1-370d8e7518d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:54.494785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:54.495986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:30:54.497039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:54.497849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 359/732] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:30:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:54.507246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04782d-c4f0-4253-a420-624ae47a09f8 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.507246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04782d-c4f0-4253-a420-624ae47a09f8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.507246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04782d-c4f0-4253-a420-624ae47a09f8 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.507246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 360/733] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:54.507623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04782d-c4f0-4253-a420-624ae47a09f8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:54.513001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80009e-a93d-475d-a6c8-5990ed0b0940 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:54.545183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:54.559217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:54.559217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.559217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.559217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.569971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:54.571430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:54.586582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:54.587652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 374/734] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:54.594059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:54.634957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04782d-c4f0-4253-a420-624ae47a09f8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:54.665148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:54.681718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f76340-8184-4934-b0cf-e8a94bea2e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:54.682413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f76340-8184-4934-b0cf-e8a94bea2e9a None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.682984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f76340-8184-4934-b0cf-e8a94bea2e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.683770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f76340-8184-4934-b0cf-e8a94bea2e9a None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.684435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f76340-8184-4934-b0cf-e8a94bea2e9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:54.686225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f76340-8184-4934-b0cf-e8a94bea2e9a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:54.706151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 361/735] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:30:54 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:54.708203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b62d32-854c-4c72-b37f-a2fa1a7f7993 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.708676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b62d32-854c-4c72-b37f-a2fa1a7f7993 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.708979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b62d32-854c-4c72-b37f-a2fa1a7f7993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.711268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02d537-a9fc-4d38-ae95-b633fbbf4729 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:54.738061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f7b62d32-854c-4c72-b37f-a2fa1a7f7993 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:54.739727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f7b62d32-854c-4c72-b37f-a2fa1a7f7993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:54.765541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:54.766184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:54.766472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:54.766753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:54.767496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:54.768394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:54.769981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:54.780960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 375/736] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:30:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:54.781970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:54.845279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 362/737] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:30:54.847038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b62d32-854c-4c72-b37f-a2fa1a7f7993 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:54.856208 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:54.951864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136e0eb-a2bc-4f52-84d3-435c41585ea0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:54.984635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b62d32-854c-4c72-b37f-a2fa1a7f7993 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.001187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:55.020132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.021032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.021744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.022176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.026765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.027418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.027704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.027979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.028729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:55.029040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:55.030543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:55.039622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:30:55.051353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:55.052650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:30:55.053516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:55.054162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 363/738] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 06:30:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:55.054162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.056211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 376/739] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:30:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:55.058862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.205115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552f32e-9f6b-43e9-9490-dfc019f24ce2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.206086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5c800-0681-473d-9666-28e8d8a87c7c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.214174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:55.244657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.245689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.245980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.246258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.258395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:55.259968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:55.268798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.269491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.269715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.269990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:55.278305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:55.279367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:55.285498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:55.285498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:55.285498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:55.285498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 364/740] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:55.285498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.319295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:55.363618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:55.365048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:55.414974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95009-960a-42d0-b322-df19fec14a60 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.456088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.456633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.456920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.457202 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.457884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:55.458184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:55.462395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:55.476380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 365/741] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:30:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:55.477267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.487467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 377/742] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:30:55.488298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.491871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:55.606657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:55.616778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a9834-2171-4220-9d2e-866edcca16fd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.628518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4aa6-18c7-4469-9dd1-8a3952becbca None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.659481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.660109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.660400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.660685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.661558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:55.661872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:55.663377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:55.671501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.673092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.673580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.674076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.675451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:55.677271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:55.678626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 378/743] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:55.679681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.701924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 366/744] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:30:55 2022] POST /identity/v3/services => generated 240 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:30:55.704187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.828471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:55.859826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6743f9-2650-4ec3-a03e-c1eed133f6e5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.901802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080d219-8d82-4aec-9054-6ab5adfef21f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:55.917746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.918878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.918878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.919366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:55.933599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:55.935114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:55.937557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:55.961966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:55.963008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:55.964057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:55.964057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:55.964057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:55.964657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 367/745] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:55.965114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:55.981778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:55.982451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:55.982740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:55.983024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.007829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.009800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:56.040441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:56.041475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:56.042418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:56.042418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:56.042418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:56.051629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 379/746] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:56.053775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:56.162250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785d9d-f8e0-41c8-b821-f7bcdd139f4f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:56.215920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:56.216580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.216863 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.220464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.234459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.235883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.239157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13ef76-4f79-4a8e-abd8-dd2b73198ad7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:56.258406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:56.259836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:30:56.260930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:56.264385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 368/747] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:30:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:56.265873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:56.269271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:56.293641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:56.294655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.295141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.295660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.296551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:56.297315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.298946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.318726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 380/748] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:56.321461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:56.365960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:56.440574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5f58-8219-41aa-812d-b71da13fc1b3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:56.446917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037006-8988-4c8c-a306-92f7bfa317fc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:56.489538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:56.490538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.491023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.491610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.492569 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:56.493078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.494930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.500413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:56.501042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.501327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.501622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.502324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:56.502629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.504035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.518009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 381/749] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:56.519284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:56.529993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 369/750] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:56.530885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:56.565231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:56.648098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc563cc-c1e6-4280-84f0-d01c7d0b2841 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:56.687402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:56.688480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.688964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.689797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.697923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.699677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.703436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717d967-c459-4406-ab73-2233e809cf05 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:56.713167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:56.714115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:56.715066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:56.717847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 382/751] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:56.718875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:56.725879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:56.752404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebd7b3-6fe4-406b-aee7-35e7e4b161b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:56.753269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd7b3-6fe4-406b-aee7-35e7e4b161b7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.753702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd7b3-6fe4-406b-aee7-35e7e4b161b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.754106 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebd7b3-6fe4-406b-aee7-35e7e4b161b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.835069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:56.847331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e9359-f1f9-43e8-b883-f61e95ea2c8c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:56.849053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebd7b3-6fe4-406b-aee7-35e7e4b161b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.851553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebd7b3-6fe4-406b-aee7-35e7e4b161b7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.894827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:56.895886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.896370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.897064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.906700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:56.908524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:30:56.922638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:56.923689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:30:56.924649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:56.935560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 383/752] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:56.937002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:56.991338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 370/753] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:30:56.991338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbaa30-c0ae-49e9-b146-49744f25c8d9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:56.991338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbaa30-c0ae-49e9-b146-49744f25c8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:56.991338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbaa30-c0ae-49e9-b146-49744f25c8d9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:56.991338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 371/754] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:56.991338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbaa30-c0ae-49e9-b146-49744f25c8d9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.005230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:57.065974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb13c89-4a36-497e-b934-ed71c019d6b9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.114199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567c6f9-4d53-4452-b0ee-266fb4a9c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.114821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c6f9-4d53-4452-b0ee-266fb4a9c617 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.115102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c6f9-4d53-4452-b0ee-266fb4a9c617 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.115415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c6f9-4d53-4452-b0ee-266fb4a9c617 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.115817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbaa30-c0ae-49e9-b146-49744f25c8d9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.116121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567c6f9-4d53-4452-b0ee-266fb4a9c617 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:57.116433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567c6f9-4d53-4452-b0ee-266fb4a9c617 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.117877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567c6f9-4d53-4452-b0ee-266fb4a9c617 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.136691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 384/755] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:57.137003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90028318-9411-4278-b342-dc8d88d15b19 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.137442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90028318-9411-4278-b342-dc8d88d15b19 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.137799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90028318-9411-4278-b342-dc8d88d15b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.158257 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-90028318-9411-4278-b342-dc8d88d15b19 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:57.159625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-90028318-9411-4278-b342-dc8d88d15b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:57.161138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.162076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.162533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.163074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.164083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:57.164586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.166201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.170977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:57.194332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 372/756] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:30:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:57.195504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.262553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 385/757] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:30:57.262553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90028318-9411-4278-b342-dc8d88d15b19 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.303427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:57.345426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46d583-f0d5-4c2f-93d6-568024ae8ae0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.390914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.392300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.393044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.393502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.401109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90028318-9411-4278-b342-dc8d88d15b19 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.408081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.411857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:57.443033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:57.444122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:57.445044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:57.445044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:57.445044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:57.451400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 373/758] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:57.452687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.462330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.462956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.463271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.463559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.464309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f7ca3818552143069b0bacbb5bec7b62) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:57.464626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.466329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.480331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 386/759] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:57 2022] GET /identity/v3/role_assignments?user.id=f7ca3818552143069b0bacbb5bec7b62&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:57.480781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.508147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:57.618960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b513ce3-e6e7-47a3-81a0-b6eb5eb0c5aa None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.646102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:57.665834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.666664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.667091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.668836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.679088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6f351-8923-4156-80b1-865cc4a35dd0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.686244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.687951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:30:57.708036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:57.709676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:30:57.710819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:30:57.710819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:30:57.710819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:57.718407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 374/760] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:57.719596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.756614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.757391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.757776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.758146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.758909 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:57.759406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.760942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.783022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 387/761] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:57.783022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.876099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa310e-21de-4894-9687-a02250efe85e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.889822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:57.918210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.919182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.919661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.920103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.921015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.922750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.931313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925210d-eac5-4754-bb20-eeeac31d6755 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:57.938031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:57.939098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:30:57.940351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:57.942787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 375/762] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 24 06:30:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:57.943816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:57.980818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06390869-7514-49f1-9d6e-d59a8ff5be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:57.981345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06390869-7514-49f1-9d6e-d59a8ff5be6a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:57.981687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06390869-7514-49f1-9d6e-d59a8ff5be6a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:57.981969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06390869-7514-49f1-9d6e-d59a8ff5be6a None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:57.982667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06390869-7514-49f1-9d6e-d59a8ff5be6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:57.982977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06390869-7514-49f1-9d6e-d59a8ff5be6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:57.984748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06390869-7514-49f1-9d6e-d59a8ff5be6a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:57.999524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:58.005273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 388/763] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:58.006791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3a2d-6691-421f-a888-325d6d0bf0ca None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.007086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3a2d-6691-421f-a888-325d6d0bf0ca None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.007443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd3a2d-6691-421f-a888-325d6d0bf0ca None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.012883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 389/764] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:58.012883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3a2d-6691-421f-a888-325d6d0bf0ca None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.083141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86de2f-d2e6-4a69-b047-fab84c1ba29b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.119010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.119717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.120246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.120527 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.130372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.132147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.133931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd3a2d-6691-421f-a888-325d6d0bf0ca None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:58.146471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:58.147715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:58.148629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:58.151435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 376/765] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:58.152752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.171505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:58.183628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363d796-33c0-4ea5-85c1-67d5ca820b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.184252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363d796-33c0-4ea5-85c1-67d5ca820b04 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.184655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363d796-33c0-4ea5-85c1-67d5ca820b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.184934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363d796-33c0-4ea5-85c1-67d5ca820b04 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.185663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363d796-33c0-4ea5-85c1-67d5ca820b04 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:58.186125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363d796-33c0-4ea5-85c1-67d5ca820b04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.188312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363d796-33c0-4ea5-85c1-67d5ca820b04 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.198656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 390/766] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:58.198656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4185-71a4-4a84-bd01-2a8fbdf39e04 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.198656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4185-71a4-4a84-bd01-2a8fbdf39e04 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.198656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab4185-71a4-4a84-bd01-2a8fbdf39e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.218920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a0ab4185-71a4-4a84-bd01-2a8fbdf39e04 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:58.220660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a0ab4185-71a4-4a84-bd01-2a8fbdf39e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:58.296563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c5a64-bf63-42f8-8fcf-fad35dee5b29 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.298310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:58.318474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 391/767] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:30:58.319382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4185-71a4-4a84-bd01-2a8fbdf39e04 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.349434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.350709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.351376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.351795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.362224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.364053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:30:58.382353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:58.383518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:30:58.384705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:30:58.387338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 377/768] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:58.388382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.469307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab4185-71a4-4a84-bd01-2a8fbdf39e04 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.522500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:58.528985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52774b-fb54-4f5d-afec-cbabdf6f7cbc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.561328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.563467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.563763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.564046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.566966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:58.567239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.571605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.575122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.576087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.576892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.577488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.578363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:58.578873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.580531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.593433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 378/769] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:58.594512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 392/770] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:58.595019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.600799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.627276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:58.715484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c4bb6-c163-4036-a557-9f83946c1d68 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.739241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadd653-d967-46aa-942c-0d8a8a03c323 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.758110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.758730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.759014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.759370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.760197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:58.760621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.762163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.783474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 379/771] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:58.784685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.787433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.788211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.788479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.788905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.789630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] RBAC: Adding query filter params (interface=public, service_id=d982ad7cc9d041ccaa05dca5ef15d46c, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:58.789947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.791628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.807877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 393/772] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/endpoints?service_id=d982ad7cc9d041ccaa05dca5ef15d46c&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:58.807877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.864449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:58.893742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:58.917241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726f35-22b6-45bb-9115-c4761f83c824 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.944108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db3acb-6345-4bdd-97a7-aca212cc05ee None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:58.966616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.967526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.968076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.969266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.970051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f2837a414b814cf7915592c7f67cbd01) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:58.971318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.971835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:58.978053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 380/773] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/role_assignments?user.id=f2837a414b814cf7915592c7f67cbd01&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:58.978839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:58.992744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:58.993460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:58.993751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:58.994084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] PATH_INFO: `/v3/services/network` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:58.994740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:58.997597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:30:59.012765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:30:59.013890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:30:59.015343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:30:59.020293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 394/774] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:30:58 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:30:59.021313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:59.191222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd6399-057d-4d3e-a7b1-b88361b2acfb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:59.194807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9b4ac-fbd6-4a45-be16-fa044327d280 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:59.200915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:59.253370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:59.254351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:59.257204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:59.257829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:59.258560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:59.259326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:59.259606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:59.259883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/8e1ec869263246f58f5a32d855a3d302/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:59.260510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:59.262141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:59.262321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=8e1ec869263246f58f5a32d855a3d302, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:59.263778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:59.272990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 395/775] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:30:59 2022] GET /identity/v3/services => generated 772 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:59.279090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:59.296079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 381/776] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:30:58 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/8e1ec869263246f58f5a32d855a3d302/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:30:59.408230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf7b00-2d3d-4212-8340-b37ed8fbf73c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:59.450096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e1c61-af6e-451f-b299-3e86cd0f1c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:59.450812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1c61-af6e-451f-b299-3e86cd0f1c9d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:59.451135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1c61-af6e-451f-b299-3e86cd0f1c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:59.451468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e1c61-af6e-451f-b299-3e86cd0f1c9d None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:59.454899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e1c61-af6e-451f-b299-3e86cd0f1c9d None admin] RBAC: Adding query filter params (name=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:59.454899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e1c61-af6e-451f-b299-3e86cd0f1c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:59.454899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e1c61-af6e-451f-b299-3e86cd0f1c9d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:59.459808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 396/777] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:30:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:59.468586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:59.495878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:59.608959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99b190-e535-4950-b54b-f377dc8af168 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:59.609337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99b190-e535-4950-b54b-f377dc8af168 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:59.609587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99b190-e535-4950-b54b-f377dc8af168 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:59.611115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 397/778] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:30:59.616557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b25d1a-5ed1-4bc8-bdab-30bb5810d76c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:59.616829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b25d1a-5ed1-4bc8-bdab-30bb5810d76c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:59.617056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b25d1a-5ed1-4bc8-bdab-30bb5810d76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:59.638065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071057d-e138-4b8a-8c12-0bf2107b1db3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:59.651642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-45b25d1a-5ed1-4bc8-bdab-30bb5810d76c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:30:59.652708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-45b25d1a-5ed1-4bc8-bdab-30bb5810d76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:30:59.707564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:59.715950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:59.717249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:59.717780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:59.718564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:59.720018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] RBAC: Adding query filter params (type=network) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:59.720551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:59.722165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:59.739000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 382/779] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:30:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:59.745674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:30:59.765056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 398/780] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:30:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:30:59.833851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:30:59.860342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d67119-f85e-4fe8-98b8-e12868d4ea43 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:30:59.907980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:30:59.908623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:30:59.908909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:30:59.909185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:30:59.909908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] RBAC: Adding query filter params (interface=public, service_id=67c7c7a1882e4cd7a28428312d7a1ddf, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:30:59.910218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:30:59.911642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:30:59.928946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 383/781] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:30:59 2022] GET /identity/v3/endpoints?service_id=67c7c7a1882e4cd7a28428312d7a1ddf&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:30:59.935533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b25d1a-5ed1-4bc8-bdab-30bb5810d76c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:00.133420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b25d1a-5ed1-4bc8-bdab-30bb5810d76c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:00.191074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:00.214263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:00.217566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:00.218256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:00.218727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:00.219526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:00.221331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:00.247539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 399/782] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:30:59 2022] GET /identity/v3/services => generated 772 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:00.313945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d9c92-44ec-4f2d-9856-4e6785e9de96 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:00.353006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:00.353006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:00.353006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:00.353006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:00.355169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:00.377573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:00.382849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:00.384135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:00.385242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:00.388532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 384/783] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:00.395244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:00.482891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:00.587175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c2ff6-4583-4031-97f3-f3f1f48ef5ca None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:00.658698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:00.659272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:00.659565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:00.659845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:00.660531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:00.660851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:00.663400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:00.681617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 400/784] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:00.688667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:00.911438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745157e-9463-4995-be96-e6cd0d9f16d9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:00.981476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:00.982183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:00.982473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:00.982753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.006377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:01.008739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:01.030772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:01.031979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:01.032890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:01.032890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:01.032890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:01.035323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 385/785] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:01.038794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:01.239117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:01.248090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8420cf-36d9-49fa-b926-f4e14eb9965c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:01.321040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce1226-a9ee-4c07-900f-e7527696cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:01.322063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce1226-a9ee-4c07-900f-e7527696cdaf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:01.322573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce1226-a9ee-4c07-900f-e7527696cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:01.323045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce1226-a9ee-4c07-900f-e7527696cdaf None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.323941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce1226-a9ee-4c07-900f-e7527696cdaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:01.324446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce1226-a9ee-4c07-900f-e7527696cdaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:01.326053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce1226-a9ee-4c07-900f-e7527696cdaf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:01.354701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 401/786] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:01.364375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:01.460941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:01.552813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566548fa-f8d3-4c15-95ea-96ed2d0d3778 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:01.553154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566548fa-f8d3-4c15-95ea-96ed2d0d3778 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:01.553376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566548fa-f8d3-4c15-95ea-96ed2d0d3778 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.554878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 402/787] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:01.560060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cee7b-5148-4bbc-995d-946008163a64 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:01.564962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771a11d-baf7-4783-997f-d7b0c71a82aa None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:01.565250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771a11d-baf7-4783-997f-d7b0c71a82aa None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:01.565525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771a11d-baf7-4783-997f-d7b0c71a82aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.607234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4771a11d-baf7-4783-997f-d7b0c71a82aa None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:01.608557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4771a11d-baf7-4783-997f-d7b0c71a82aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:01.627776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:01.629314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:01.629314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:01.629314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.650134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:01.652106 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:01.675065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:01.676129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:31:01.677145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:01.682736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 386/788] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:01.686872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:01.771943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 403/789] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:01.842195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ec0ab-84a5-4bea-909a-cd9a06cde94d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:01.842497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ec0ab-84a5-4bea-909a-cd9a06cde94d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:01.842707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ec0ab-84a5-4bea-909a-cd9a06cde94d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.854525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761f868-df38-4eef-bf02-3f7a7d1e659e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:01.856469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 404/790] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:01 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:01.865113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:01.866003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdfeb-44aa-4a30-8063-9ca0b207ceb9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:01.866602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdfeb-44aa-4a30-8063-9ca0b207ceb9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:01.866602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdfeb-44aa-4a30-8063-9ca0b207ceb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.894714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:01.895752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:01.896227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:01.896712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:01.897639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:01.898135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:01.900232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:01.916940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5d6fdfeb-44aa-4a30-8063-9ca0b207ceb9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:01.918171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5d6fdfeb-44aa-4a30-8063-9ca0b207ceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:01.923368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 387/791] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 06:31:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:01.936528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.025325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 405/792] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:02.096331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb31870-7829-48e0-b8bf-1f4e2760bade None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:02.101881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f671-4d7a-4bca-8d13-0efec06a75ef None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.102177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f671-4d7a-4bca-8d13-0efec06a75ef None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.102461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f671-4d7a-4bca-8d13-0efec06a75ef None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.104073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 406/793] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:02.110214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b795-27de-423f-905b-c375c1368cb2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.110504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b795-27de-423f-905b-c375c1368cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.110788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b795-27de-423f-905b-c375c1368cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.130544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7389b795-27de-423f-905b-c375c1368cb2 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:02.131770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7389b795-27de-423f-905b-c375c1368cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:02.137068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:02.137719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.138004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.138278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.147292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:02.154457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:02.156084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:02.177756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:02.178756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:02.179707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:02.179707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:02.179707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:02.183627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 388/794] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:02.188590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.236208 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:02.241210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 407/795] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:02.247587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389b795-27de-423f-905b-c375c1368cb2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.396311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffec9-3a06-450c-a9a3-08d7e86ede4b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:02.440208 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:02.441266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.441691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.442989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.443850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:02.444300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:02.445847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:02.462495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 389/796] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:02.463606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.485017 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389b795-27de-423f-905b-c375c1368cb2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:02.565702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:02.565937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:02.566359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.566647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.566920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.581834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:02.587170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:02.588324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:31:02.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:02.594653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e256711-03b3-46ad-a14b-15748d9f7c16 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:02.596856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 408/797] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:31:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:02.596856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.602121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:02.643755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:02.645368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.645658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.645932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.646588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:02.647973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:02.661547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:02.662625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:02.664267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 390/798] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Aug 24 06:31:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:02.665111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.742182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba981ae-515e-4482-a10b-48ed140a4626 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:02.785495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:02.786155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.786439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.786720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.797756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:02.798042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6e5b0-8ac7-42b5-b9ca-d43d909a11f1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:02.799851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:02.813697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:02.815091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:02.816060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:02.818535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 409/799] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:02.819597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.838770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:02.839407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:02.839685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:02.840145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:02.854333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:02.861329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:02.862362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:02.863289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 391/800] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:02.864018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:02.869536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:02.905157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:02.960706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780687-cb14-408e-90ae-d1c27873a6d7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.001350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.001967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.002248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.002520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.003255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:03.003566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.003957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66522b5-0beb-4b58-941e-b18565d96b6c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.004935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.015392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 410/801] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:31:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:03.015392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.044073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.044979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.045261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.045534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.046254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:03.046776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.048394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.053150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 392/802] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 06:31:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:03.053939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.135140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b199d-4ea7-4f37-9dad-7a5e1fc7edc3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.165268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:03.175341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4e83f-e1a7-41b4-b94c-d748cf5adedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.175980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4e83f-e1a7-41b4-b94c-d748cf5adedf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.176269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4e83f-e1a7-41b4-b94c-d748cf5adedf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.176550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4e83f-e1a7-41b4-b94c-d748cf5adedf None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.177263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4e83f-e1a7-41b4-b94c-d748cf5adedf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:03.177576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4e83f-e1a7-41b4-b94c-d748cf5adedf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.179079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4e83f-e1a7-41b4-b94c-d748cf5adedf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.194808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 411/803] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:03.196128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b913e3-c261-4203-8e2a-637e077617f2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.196469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b913e3-c261-4203-8e2a-637e077617f2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.196693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b913e3-c261-4203-8e2a-637e077617f2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.202351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 412/804] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:03.202351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b913e3-c261-4203-8e2a-637e077617f2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.247804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:03.284518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469daf0-d508-4c2a-b616-fe9e51ae0c84 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.319032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b913e3-c261-4203-8e2a-637e077617f2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.323138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.323898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.324262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.324582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.325378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:03.325746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.327171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.336597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 393/805] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:31:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:03.337596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.362229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.362873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.363156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.363481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.373672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.375237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:03.394948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:03.396097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:03.397026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:03.397026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:03.397026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:03.397026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 413/806] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:03.397249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.443375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:03.454418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5414c3-63e4-4b1a-ad49-cf27efa20787 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.482676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:03.493730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b5b7d-162f-42a5-882f-a7579cc0f5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.494350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b5b7d-162f-42a5-882f-a7579cc0f5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.494676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b5b7d-162f-42a5-882f-a7579cc0f5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.494956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b5b7d-162f-42a5-882f-a7579cc0f5a9 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.495652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b5b7d-162f-42a5-882f-a7579cc0f5a9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:03.495959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b5b7d-162f-42a5-882f-a7579cc0f5a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.497370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b5b7d-162f-42a5-882f-a7579cc0f5a9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.502578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 394/807] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Aug 24 06:31:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:03.504252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a9be-521f-449e-8b78-380736fddb0f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.504702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a9be-521f-449e-8b78-380736fddb0f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.505256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903a9be-521f-449e-8b78-380736fddb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.513513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fe92d-abd2-440d-9055-9a2db32a1899 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.523078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3903a9be-521f-449e-8b78-380736fddb0f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:03.524316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3903a9be-521f-449e-8b78-380736fddb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:03.553645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.554646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.554928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.555221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f7ca3818552143069b0bacbb5bec7b62/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.557210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=f7ca3818552143069b0bacbb5bec7b62, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.558658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.573482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 414/808] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:31:03 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f7ca3818552143069b0bacbb5bec7b62/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:31:03.574176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.606538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 395/809] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:03.607687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a9be-521f-449e-8b78-380736fddb0f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.735442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:03.743809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dc4dd-d6bb-4356-b8aa-ad37cd39071f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.791973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903a9be-521f-449e-8b78-380736fddb0f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:03.818583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:03.819934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.820591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.821141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.821141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.837777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.839365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.840820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:03.841729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:03.842243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:03.842818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:03.843835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:03.844348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:03.845937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:03.865546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 396/810] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:03.866765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:03.876296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:03.877263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:03.878169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:03.878169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:03.878169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:03.878169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 415/811] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:03.880880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.004544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd13ec-80ca-445e-b1a7-f4bc67074613 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.034589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49941580-49b8-4b01-836d-6eada3de62cb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.051795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.053125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.053410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.059215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:04.059215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:04.059215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:04.083224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.083224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.083224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.083224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:04.099217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:04.099217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:04.107221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 397/812] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:04.107221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:04.115225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:04.116282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:04.117254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:04.123221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 416/813] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:31:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:04.123221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.138445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:04.175218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:04.289268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5008a9-afa2-4eed-9749-fc60914c4f5d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.324311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d8f31-4cb7-4027-996a-7c3e1b261a55 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.344412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.346043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.346344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.346670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] PATH_INFO: `/v3/services/network` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:04.347311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:04.349222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:04.361556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:04.362573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:04.363467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 417/814] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:04 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:04.364288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.378721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.380412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.380801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.381139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:04.382178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:04.382792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:04.384816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:04.402662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 398/815] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:04.403805 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.551324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:04.576361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:04.584474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa071e5-8b0f-4f37-83bf-fed25d10e16d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.595228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d1a61-fa96-4500-ba08-91f8a893973c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.625031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.626126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.626603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.627069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:04.629294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:04.629790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:04.633278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:04.659243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 418/816] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 06:31:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:04.660458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.687978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.688563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.688849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.689124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:04.690244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] RBAC: Adding query filter params (name=network) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:04.690675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:04.692704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:04.711163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 399/817] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:04.711163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.786621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa69d81-39cd-4038-9360-769d89b4b92c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.820932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:04.827915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.828866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.829360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.829825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:04.837730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:04.839422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:04.851597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:04.852617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:04.853586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:04.856290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 419/818] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:31:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:04.857346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:04.900678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77005338-39de-48ad-b78a-20fe29d2030e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.976695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272a813-0245-460e-b8b5-6f296ad2f253 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:04.976917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:04.977558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:04.977841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:04.978116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.004490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:05.005966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:05.018446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:05.019404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:05.019898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:05.020377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.021294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] RBAC: Adding query filter params (type=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:05.021805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:05.023404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:05.034019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 420/819] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:05.034459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:05.035618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:05.036641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:05.036641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:05.036641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:05.037941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 400/820] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:05.038222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:05.040182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:05.066063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:05.110757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:05.158008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79616-3b4e-4107-b2d2-c5a112e47c30 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:05.196590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:05.197577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:05.198080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:05.198542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.199547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:05.200049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:05.201695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:05.229439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 421/821] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:31:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:05.230599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:05.246115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965429-a339-45c5-b5aa-871f9b862e33 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:05.313854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:05.314927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:05.315236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:05.315513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.316187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:05.321026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:05.375350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 401/822] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:05.375350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:05.380847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb2929-270c-4d45-8a4d-22f139ea8cdb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:05.405722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:05.461225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:05.462242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:05.462737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:05.471221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.472478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:05.472987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:05.474597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:05.497363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 422/823] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:05.507222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:05.547226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:05.683644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cae03d-583f-4a57-bc69-4cb28cb5ec58 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:05.746843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a6cb-6fe5-4077-a8ef-8fcb6f52fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:05.747291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a6cb-6fe5-4077-a8ef-8fcb6f52fce3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:05.747590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a6cb-6fe5-4077-a8ef-8fcb6f52fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:05.747976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a6cb-6fe5-4077-a8ef-8fcb6f52fce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.753613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8685-6ee8-47a0-a444-e6a7ed2667de None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:05.800771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5978cf-e00d-4b3a-9a5a-db9bbc02dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:05.802759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5978cf-e00d-4b3a-9a5a-db9bbc02dc6e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:05.803659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5978cf-e00d-4b3a-9a5a-db9bbc02dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:05.803947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5978cf-e00d-4b3a-9a5a-db9bbc02dc6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.811544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5978cf-e00d-4b3a-9a5a-db9bbc02dc6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:05.813279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5978cf-e00d-4b3a-9a5a-db9bbc02dc6e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6c5978cf-e00d-4b3a-9a5a-db9bbc02dc6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:05.827797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:05.828828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:05.829764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 423/824] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:05.830293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024c71b-9bfd-4ef0-acc4-fac8ac7318dc None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:05.830945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024c71b-9bfd-4ef0-acc4-fac8ac7318dc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:05.831412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024c71b-9bfd-4ef0-acc4-fac8ac7318dc None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:05.837383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 424/825] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:05.838244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024c71b-9bfd-4ef0-acc4-fac8ac7318dc None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:05.851852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53a6cb-6fe5-4077-a8ef-8fcb6f52fce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:05.853918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53a6cb-6fe5-4077-a8ef-8fcb6f52fce3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:05.884101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:05.953150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:05.971312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024c71b-9bfd-4ef0-acc4-fac8ac7318dc None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.011920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.013667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.014786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.015423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.017503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:06.018849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:06.027265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:06.027699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 402/826] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:05 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:06.029500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443cdc8-a6e3-4b61-867a-d017ff190882 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.029938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443cdc8-a6e3-4b61-867a-d017ff190882 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.030502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443cdc8-a6e3-4b61-867a-d017ff190882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.051177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 425/827] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:06.052173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.064107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4443cdc8-a6e3-4b61-867a-d017ff190882 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:06.066388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4443cdc8-a6e3-4b61-867a-d017ff190882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:06.243235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 403/828] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:06.243235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443cdc8-a6e3-4b61-867a-d017ff190882 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.300415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:06.311154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152faea8-1f1b-42be-9c17-5bdd405a2742 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.353087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42a1be-a337-4e4f-9083-c2f4fdced3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.353716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42a1be-a337-4e4f-9083-c2f4fdced3bc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.353992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42a1be-a337-4e4f-9083-c2f4fdced3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.354261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42a1be-a337-4e4f-9083-c2f4fdced3bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.365031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42a1be-a337-4e4f-9083-c2f4fdced3bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:06.365548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:06.366621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42a1be-a337-4e4f-9083-c2f4fdced3bc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:06.373401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443cdc8-a6e3-4b61-867a-d017ff190882 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-3a42a1be-a337-4e4f-9083-c2f4fdced3bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:06.380643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:06.382043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:06.382939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:06.382939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:06.382939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:06.382939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 426/829] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:06.382939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f815f-a2df-43d5-a46e-5c173adbe197 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.383262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f815f-a2df-43d5-a46e-5c173adbe197 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.388499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f815f-a2df-43d5-a46e-5c173adbe197 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.388499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 427/830] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:06.389569 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f815f-a2df-43d5-a46e-5c173adbe197 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.419521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6807-f41e-4cb1-a92e-0653207b93cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.420244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6807-f41e-4cb1-a92e-0653207b93cb None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.420535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6807-f41e-4cb1-a92e-0653207b93cb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.420894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6807-f41e-4cb1-a92e-0653207b93cb None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f2837a414b814cf7915592c7f67cbd01/roles/72fda378772c45f4b41c5f973cc7cda2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.437938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de6807-f41e-4cb1-a92e-0653207b93cb None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=f2837a414b814cf7915592c7f67cbd01, role_id=72fda378772c45f4b41c5f973cc7cda2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:06.439419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de6807-f41e-4cb1-a92e-0653207b93cb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:06.463220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 404/831] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:31:06 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f2837a414b814cf7915592c7f67cbd01/roles/72fda378772c45f4b41c5f973cc7cda2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:31:06.463220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b435444-ec27-40cc-bcaa-2fc9da961dfe None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.463220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b435444-ec27-40cc-bcaa-2fc9da961dfe None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.463220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b435444-ec27-40cc-bcaa-2fc9da961dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.483805 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9b435444-ec27-40cc-bcaa-2fc9da961dfe None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:06.485294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9b435444-ec27-40cc-bcaa-2fc9da961dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:06.508685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f815f-a2df-43d5-a46e-5c173adbe197 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.557413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.558038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.558322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.558770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.559411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:06.559655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:06.561154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:06.563934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 405/832] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:06.568320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b435444-ec27-40cc-bcaa-2fc9da961dfe None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.580180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 428/833] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:06.581324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.584595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:06.679889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:06.704237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b435444-ec27-40cc-bcaa-2fc9da961dfe None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.710386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631c4ba-aab5-40d5-bb62-4bf741d520f1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.741279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.742003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.742390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.742813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.747431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.748171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.748457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.748743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.754534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:06.756187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:06.757732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:06.759812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:06.760938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:06.761986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:06.764968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 406/834] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:06.770378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:06.776820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:06.778249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:06.779576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:06.780763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 429/835] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:06.786832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.862675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:06.894350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179a64e-d1a2-493e-8b3e-460423cad0fb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.906690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248e30a-8d25-47e9-8d51-18ea85c7fcfe None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:06.930187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.930791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.931089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.931379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.932117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:06.932427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:06.933890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:06.936669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:06.944007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 407/836] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:06.944835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:06.950745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:06.951433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:06.951716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:06.951965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:06.953201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:06.953201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:06.954626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:06.967297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 430/837] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:06.968360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:07.060881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f09a87-4fd8-477c-b9a9-4e5725575fcf None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.090320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7171e7-80d1-4124-b96f-f5b2911ec07e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.103169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.104217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.104699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.105471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.106509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.109181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.114215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:07.120206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:07.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:07.127183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 408/838] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:07.128095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:07.135894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.136858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.137146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.137573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.148432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.149899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:07.164874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:07.165854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:07.166745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:07.166745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:07.166745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:07.166745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 431/839] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:07.166745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:07.195175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:07.238144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6667b5-a80e-4c6a-a59d-134527978376 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.276251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.276949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.277132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.277403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.278292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=8e1ec869263246f58f5a32d855a3d302) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:07.278443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.279875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.281727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f327-e436-489b-9307-1e8652024332 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.287338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 409/840] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:07 2022] GET /identity/v3/role_assignments?user.id=8e1ec869263246f58f5a32d855a3d302&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:07.288481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:07.371318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.371318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.371318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.371318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.371875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:07.375230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.379218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.389239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 432/841] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:07.394483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:07.481241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:07.504242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ed294-f504-4838-bdf8-448246bae6be None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.540614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:07.551008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.551683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.551963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.552235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.552891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:07.553190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.554557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.580298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 410/842] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:07.580298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:07.582750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74dc9b-5b1e-4bd3-b195-b7bd0c1f06a6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.631500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.632500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.632962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.633423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.634307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:07.634802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.636495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.643029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 433/843] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:07.710229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932401c-3d9c-440e-9af3-6f0c9d56f890 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.750682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.751376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.751661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.751933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.760190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.761663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:07.775027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:07.776123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:31:07.777119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:07.782043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 411/844] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:07.788497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:07.910387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f84b-f333-42d5-81c1-a52edbefbc01 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:07.911875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:07.946124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:07.946823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:07.947167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:07.947531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:07.948333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:07.948703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:07.950192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:07.969004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 434/845] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Aug 24 06:31:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:07.974602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:08.024403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:08.082301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa7884-eabf-4fd9-b35a-ef9a77feb673 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:08.117720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943753-07a1-43b9-9f7a-bd33ab662a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:08.118336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943753-07a1-43b9-9f7a-bd33ab662a1d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:08.118620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943753-07a1-43b9-9f7a-bd33ab662a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:08.118893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943753-07a1-43b9-9f7a-bd33ab662a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:08.194738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943753-07a1-43b9-9f7a-bd33ab662a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:08.196143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943753-07a1-43b9-9f7a-bd33ab662a1d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:08.243566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:08.303310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 412/846] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:08.305076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:08.408626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d6025-79a5-44a8-8067-bf6fab9a128f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:08.444443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:08.445394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:08.445825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:08.446271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:08.456887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:08.458623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:08.460831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b5cf-5528-4a9d-a22f-4c1852c7800b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:08.461239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b5cf-5528-4a9d-a22f-4c1852c7800b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:08.461791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b5cf-5528-4a9d-a22f-4c1852c7800b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:08.466997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 413/847] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:08.468563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d7f37-5384-4d45-bea2-78c4fd24ec53 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:08.468843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d7f37-5384-4d45-bea2-78c4fd24ec53 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:08.469115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d7f37-5384-4d45-bea2-78c4fd24ec53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:08.476274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:08.477209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:08.478156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:08.478156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:08.478156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:08.478156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 435/848] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:08.484869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:08.494800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5d4d7f37-5384-4d45-bea2-78c4fd24ec53 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:08.496452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5d4d7f37-5384-4d45-bea2-78c4fd24ec53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:08.502091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:08.583489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 414/849] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:08.607166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaebf3-ab8f-43ab-a63f-1c748b58385d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:08.643735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:08.644723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:08.645232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:08.645683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:08.646505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:08.646985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:08.648619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:08.666504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 436/850] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:08.669045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d7f37-5384-4d45-bea2-78c4fd24ec53 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:08.751857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:08.777034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d7f37-5384-4d45-bea2-78c4fd24ec53 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:08.814445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:08.815807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:08.816267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:08.816730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:08.824317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:08.826027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:08.843093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:08.844733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:08.846298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:08.851538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 415/851] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:08.854333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:08.856008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:08.952263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:08.985151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce0398-e305-4a3f-b453-9803759c8e92 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:08.993347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd40a-d5c4-4bec-8b4a-b6cf886fe65c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.029406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.030285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.030716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.031231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.031644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:09.031954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:09.033848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:09.037562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:09.041022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.043166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.043166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.043166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.050826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 416/852] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:09.059113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:09.060627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:09.065851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:09.066815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:09.068379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 437/853] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:31:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:09.072832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:09.176695 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99420d-8bde-45f3-8283-0e81eb7225d9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.213865 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2863106-8343-4a1b-ac7f-e2fc9cfb68e2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.228394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.229021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.229305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.229643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.230389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f7ca3818552143069b0bacbb5bec7b62) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:09.230935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:09.232934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:09.247307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 417/854] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:09 2022] GET /identity/v3/role_assignments?user.id=f7ca3818552143069b0bacbb5bec7b62&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:09.258263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:09.263862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc5cbb-9517-4573-81a9-7260be51d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.264570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5cbb-9517-4573-81a9-7260be51d7ad None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.264890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5cbb-9517-4573-81a9-7260be51d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.265290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5cbb-9517-4573-81a9-7260be51d7ad None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.266199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc5cbb-9517-4573-81a9-7260be51d7ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:09.266516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc5cbb-9517-4573-81a9-7260be51d7ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:09.268017 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc5cbb-9517-4573-81a9-7260be51d7ad None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:09.278753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 438/855] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:31:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:09.285042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:09.361668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:09.400735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d16af-17f3-45a3-b405-1f91b9157ab4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.401073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d16af-17f3-45a3-b405-1f91b9157ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.401305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d16af-17f3-45a3-b405-1f91b9157ab4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.402991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 439/856] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:09.408772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45a531-856e-427c-b06b-89723d5df9fd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.409399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45a531-856e-427c-b06b-89723d5df9fd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.409399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45a531-856e-427c-b06b-89723d5df9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.412614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dbd0c-0c38-4919-98c5-e0c6707cbb1e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.428554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0e45a531-856e-427c-b06b-89723d5df9fd None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:09.429750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0e45a531-856e-427c-b06b-89723d5df9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:09.464279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.464925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.465211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.465485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.484577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:09.486867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:09.506307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:09.507641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:09.508562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:09.508562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:09.508562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:09.510488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 418/857] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:09.516021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:09.520042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 440/858] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:09.621080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe4e03-222f-4440-9171-4cb83047305e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.635548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:09.658139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfd8aa-2d60-4d04-8a55-0e4aeb64c4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.658771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd8aa-2d60-4d04-8a55-0e4aeb64c4eb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.659057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd8aa-2d60-4d04-8a55-0e4aeb64c4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.659372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd8aa-2d60-4d04-8a55-0e4aeb64c4eb None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.660039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfd8aa-2d60-4d04-8a55-0e4aeb64c4eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:09.660346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfd8aa-2d60-4d04-8a55-0e4aeb64c4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:09.661732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfd8aa-2d60-4d04-8a55-0e4aeb64c4eb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:09.674740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 419/859] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:09.680370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45a531-856e-427c-b06b-89723d5df9fd None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:09.741976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d3b88-3b5e-49ec-942c-14cb9caa2776 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.742749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d3b88-3b5e-49ec-942c-14cb9caa2776 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.743600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d3b88-3b5e-49ec-942c-14cb9caa2776 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.745810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 420/860] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:09.751045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411accb3-8f92-462b-b101-d745e44e2275 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.751379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411accb3-8f92-462b-b101-d745e44e2275 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.751658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411accb3-8f92-462b-b101-d745e44e2275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.770616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-411accb3-8f92-462b-b101-d745e44e2275 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:09.771935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-411accb3-8f92-462b-b101-d745e44e2275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:09.791586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45a531-856e-427c-b06b-89723d5df9fd None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.819131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:09.831145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.831838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:09.832113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:09.832389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:09.840162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:09.841622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:09.846039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 421/861] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:09.846814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411accb3-8f92-462b-b101-d745e44e2275 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:09.862147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:09.863206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:09.864170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:09.869450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 441/862] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:31:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:09.870382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:09.960287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411accb3-8f92-462b-b101-d745e44e2275 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.962107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:09.982439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ea0b8-e735-4c9e-ae6c-69dc12d24b22 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:09.998552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:09.999617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.000129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.000609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.001399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.002991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:10.012305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:10.013327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:10.014250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:10.017100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 422/863] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 24 06:31:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:10.022482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:10.023423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.024058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.024343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.024634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.025367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:10.025676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.027124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.050279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 442/864] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 06:31:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:10.051275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:10.096214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:10.131836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd87d1d-cfaa-4d41-a9a4-eabbed15ac6c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:10.157316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f933d7-b039-4b45-b21b-9f4a6647a95c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:10.172031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.172826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.173167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.173825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.174606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:10.175034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.176610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.186657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 423/865] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:10.186657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:10.196057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.197087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.197384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.197663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.224671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:10.279495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.281948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.310412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa6338-65ed-41da-b06c-50115dbb9e56 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:10.334788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:10.347687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.348462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.348746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.349023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.349677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.351156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.379221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 424/866] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:31:10 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:10.379221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:10.401933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 443/867] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:10.409206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:10.523771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:10.544654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd8569-6e5d-478c-ba3b-10a66c38be37 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:10.560070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec96d8-0afa-4bea-a8a8-298bacf58a64 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:10.587647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.588274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.588569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.588844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.589489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:10.590477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.591177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.596629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 425/868] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:10.608019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.609097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.609581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.610105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.623074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.625495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.628089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:10.646443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:10.647559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:10.648493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:10.648493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:10.648493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:10.650902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 444/869] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:10.656841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:10.775528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b17ce-5db2-4b61-b40e-a2f739a21273 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:10.812278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.813310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.813820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.814294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.815542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:10.816070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.817645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:10.830984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 426/870] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:10.836213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:10.943501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7ab77-3dd9-44da-83c8-709cc36578aa None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:10.981503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:10.982529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:10.983078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:10.983649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:10.991337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:10.993122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:11.005096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:11.006041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:11.006963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:11.009345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 445/871] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:11.015404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:11.027798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:11.126872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424fc3a-5aff-4c84-b237-03fa976b4fe4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:11.162917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294986e-0361-476b-8a85-709831da0dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:11.163876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294986e-0361-476b-8a85-709831da0dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:11.164360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294986e-0361-476b-8a85-709831da0dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:11.164800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294986e-0361-476b-8a85-709831da0dc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:11.165695 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294986e-0361-476b-8a85-709831da0dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:11.166147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294986e-0361-476b-8a85-709831da0dc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:11.167761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294986e-0361-476b-8a85-709831da0dc0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:11.180632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 427/872] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:11.186859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:11.250580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8d1dc-477d-4adf-8536-83b7a24dea07 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:11.251105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8d1dc-477d-4adf-8536-83b7a24dea07 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:11.251184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8d1dc-477d-4adf-8536-83b7a24dea07 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:11.252783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 428/873] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:11.257912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303062-441a-4c52-90f3-10b272e99fee None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:11.258200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303062-441a-4c52-90f3-10b272e99fee None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:11.258479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09303062-441a-4c52-90f3-10b272e99fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:11.259532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:11.278689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-09303062-441a-4c52-90f3-10b272e99fee None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:11.279856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-09303062-441a-4c52-90f3-10b272e99fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:11.305449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bd5a4-4ca8-4b05-9b33-030d7b0407f1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:11.339699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:11.340309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:11.340648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:11.340930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:11.341577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f2837a414b814cf7915592c7f67cbd01) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:11.341883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:11.343604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:11.348635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:11.355229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 446/874] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:11 2022] GET /identity/v3/role_assignments?user.id=f2837a414b814cf7915592c7f67cbd01&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:11.359788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 429/875] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:11.625204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:11.728040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cfced-8287-46b9-b983-91e06f7397fd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:11.747417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:11.765556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:11.766202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:11.766482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:11.766749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:11.767370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:11.768754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:11.777455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:11.778435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:11.779809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 447/876] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:11.784729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303062-441a-4c52-90f3-10b272e99fee None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:11.891354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09303062-441a-4c52-90f3-10b272e99fee None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:11.926403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd0772-ae78-4a75-bb39-2535474a566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:11.927024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd0772-ae78-4a75-bb39-2535474a566a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:11.927339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd0772-ae78-4a75-bb39-2535474a566a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:11.927619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd0772-ae78-4a75-bb39-2535474a566a None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:11.928263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd0772-ae78-4a75-bb39-2535474a566a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:11.928570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd0772-ae78-4a75-bb39-2535474a566a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:11.929963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd0772-ae78-4a75-bb39-2535474a566a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:11.935063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 430/877] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:11.941264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:12.050555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e078305-5aaf-4e7b-b716-352387bb8153 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:12.086447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5271f0-46f6-4ea8-bff2-8cd5365e5018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:12.087065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5271f0-46f6-4ea8-bff2-8cd5365e5018 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:12.087372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5271f0-46f6-4ea8-bff2-8cd5365e5018 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:12.087644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5271f0-46f6-4ea8-bff2-8cd5365e5018 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:12.088281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5271f0-46f6-4ea8-bff2-8cd5365e5018 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:12.088582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5271f0-46f6-4ea8-bff2-8cd5365e5018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:12.089958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5271f0-46f6-4ea8-bff2-8cd5365e5018 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:12.095402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 448/878] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:12.492534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880f83-b6d6-4e2c-87dd-e54c8e6857c3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:12.493512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880f83-b6d6-4e2c-87dd-e54c8e6857c3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:12.494056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74880f83-b6d6-4e2c-87dd-e54c8e6857c3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:12.495930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 431/879] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:12.502401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db74c-0f2e-4316-800d-33b37185f14a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:12.502946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db74c-0f2e-4316-800d-33b37185f14a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:12.503472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0db74c-0f2e-4316-800d-33b37185f14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:12.522444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7e0db74c-0f2e-4316-800d-33b37185f14a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:12.524049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7e0db74c-0f2e-4316-800d-33b37185f14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:12.571903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:12.597670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 449/880] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:12.628857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047631e9-ff18-4e34-b3a1-488564f1dd28 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:12.629478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047631e9-ff18-4e34-b3a1-488564f1dd28 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:12.629986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047631e9-ff18-4e34-b3a1-488564f1dd28 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:12.631974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 432/881] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:12.641597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94192cd-9ac8-4bc6-bf81-aeafd34191e7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:12.642494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94192cd-9ac8-4bc6-bf81-aeafd34191e7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:12.642970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94192cd-9ac8-4bc6-bf81-aeafd34191e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:12.662491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f94192cd-9ac8-4bc6-bf81-aeafd34191e7 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:12.663717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f94192cd-9ac8-4bc6-bf81-aeafd34191e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:12.733617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 450/882] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:12.874754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047631e9-ff18-4e34-b3a1-488564f1dd28 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:12.942244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:12.980785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047631e9-ff18-4e34-b3a1-488564f1dd28 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:13.009488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94192cd-9ac8-4bc6-bf81-aeafd34191e7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:13.016710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:13.017357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.017642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.017981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.019079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:13.021105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:13.032822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:13.033855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:13.034718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 433/883] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:13.041254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:13.138133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94192cd-9ac8-4bc6-bf81-aeafd34191e7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:13.146268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:13.166798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39516ff9-089e-404f-bed3-7d7f3185bb94 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:13.180808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:13.181567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.181856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.182138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.183085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:13.185296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:13.199875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:13.211630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 451/884] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:31:13 2022] POST /identity/v3/services => generated 269 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:31:13.237543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb35acc-2fab-4471-81a4-309d5ba02f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:13.238239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb35acc-2fab-4471-81a4-309d5ba02f72 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.238529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb35acc-2fab-4471-81a4-309d5ba02f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.238865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb35acc-2fab-4471-81a4-309d5ba02f72 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.239566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb35acc-2fab-4471-81a4-309d5ba02f72 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:13.239877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb35acc-2fab-4471-81a4-309d5ba02f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:13.241418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb35acc-2fab-4471-81a4-309d5ba02f72 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:13.247917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 434/885] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:13.254182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:13.371664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2d6fe-0167-4506-ab6a-af1835a51ab7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:13.413945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:13.414915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.415432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.415939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.416777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:13.417281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:13.418858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:13.425177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 452/886] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:13.428047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbcee0-14ed-4ab9-b4ba-bcf32a412929 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.428047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbcee0-14ed-4ab9-b4ba-bcf32a412929 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.428047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbcee0-14ed-4ab9-b4ba-bcf32a412929 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.429437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 435/887] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:13.431337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:13.434734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542493a-dc0d-4f4e-bc54-5a3d54ae8b15 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.435024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542493a-dc0d-4f4e-bc54-5a3d54ae8b15 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.436353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542493a-dc0d-4f4e-bc54-5a3d54ae8b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.460694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a542493a-dc0d-4f4e-bc54-5a3d54ae8b15 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:13.461897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a542493a-dc0d-4f4e-bc54-5a3d54ae8b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:13.465129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:13.548634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 436/888] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:13.559439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9af3-742d-4081-8058-eb9c3f7044d8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:13.599614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:13.600289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.600576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.600853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.601561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] RBAC: Adding query filter params (interface=public, service_id=3591d77cdf7b4befbad526e09f151729, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:13.601869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:13.603283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:13.609600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 453/889] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:31:13 2022] GET /identity/v3/endpoints?service_id=3591d77cdf7b4befbad526e09f151729&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:13.614473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542493a-dc0d-4f4e-bc54-5a3d54ae8b15 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:13.721803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542493a-dc0d-4f4e-bc54-5a3d54ae8b15 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:13.740355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:13.758373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:13.759356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.759872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.760365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:13.761189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:13.762759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:13.769249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 437/890] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:31:13 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:31:13.836034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:13.946007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:13.955382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1658f-c92e-43cd-897b-ed7d64bc0544 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:13.991566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd2a88-7103-460f-8364-752882e88a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:13.992123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd2a88-7103-460f-8364-752882e88a37 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:13.992400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd2a88-7103-460f-8364-752882e88a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:13.992672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd2a88-7103-460f-8364-752882e88a37 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.003569 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd2a88-7103-460f-8364-752882e88a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-fbcd2a88-7103-460f-8364-752882e88a37 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:14.008646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:14.009591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:14.010443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 454/891] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 06:31:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:14.017490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:14.051000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc8c3-6c7e-42d1-92e8-0f7abfe52d9f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.051342 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc8c3-6c7e-42d1-92e8-0f7abfe52d9f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.051600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc8c3-6c7e-42d1-92e8-0f7abfe52d9f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.052956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 455/892] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:14.057989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a393a7d-7f89-41c8-bd4f-333671dd2e25 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.058279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a393a7d-7f89-41c8-bd4f-333671dd2e25 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.058558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a393a7d-7f89-41c8-bd4f-333671dd2e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.082117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0a393a7d-7f89-41c8-bd4f-333671dd2e25 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:14.083975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0a393a7d-7f89-41c8-bd4f-333671dd2e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:14.124940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a990d31-5762-4ec6-bfd6-33479a8c9dcf None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:14.168416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:14.169856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.170381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.170910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.172013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:14.172854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:14.174611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:14.184545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 438/894] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 06:31:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:14.189148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 456/894] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:14.190836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a393a7d-7f89-41c8-bd4f-333671dd2e25 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:14.286312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:14.302777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a393a7d-7f89-41c8-bd4f-333671dd2e25 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:14.338015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:14.338648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.338974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.339275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.349858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:14.351349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:14.366332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:14.367347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:14.368660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:14.368660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:14.368660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:14.368660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 457/895] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:14.373774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:14.421798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:14.453597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:14.479764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10c21a-db03-4e48-8ca3-76e218023522 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:14.517865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:14.518934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.519471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.519945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.520796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:14.521319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:14.522895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:14.540215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 439/896] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:14.541429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:14.577292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127e7bb-bc24-4640-83fe-213c4e2c4840 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:14.630845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:14.631509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.631794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.632081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.632892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:14.634597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:14.639659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:14.652172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2473f-18e5-41c8-abf3-7aa88bcea74d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:14.656443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:14.659241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 458/897] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:31:14 2022] POST /identity/v3/services => generated 241 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:14.715700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:14.716677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.717099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.717537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.726790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:14.728551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:14.744693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:14.745975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:14.747212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:14.750185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 440/898] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:31:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:14.756239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:14.869910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae27124-c866-425d-836b-5cb5742279cd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:14.907141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:14.908039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:14.908501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:14.908925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:14.909810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:14.910272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:14.911882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:14.930822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 459/899] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 06:31:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:14.936663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:15.049214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037fdba-e865-4bab-8dfa-460804379328 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:15.084666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38149d7a-f3c9-47d6-a7e8-0dbe350cd1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:15.085522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149d7a-f3c9-47d6-a7e8-0dbe350cd1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.086051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149d7a-f3c9-47d6-a7e8-0dbe350cd1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.086463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149d7a-f3c9-47d6-a7e8-0dbe350cd1a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.115321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:15.159642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38149d7a-f3c9-47d6-a7e8-0dbe350cd1a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:15.161342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38149d7a-f3c9-47d6-a7e8-0dbe350cd1a8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:15.265732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 441/900] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:14 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:15.272428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:15.332057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:15.370785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8de185-7d17-4b62-96e9-eedf18d7768a None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.372800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8de185-7d17-4b62-96e9-eedf18d7768a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.373409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8de185-7d17-4b62-96e9-eedf18d7768a None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.380044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 442/901] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:15.381856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c365c1-4abb-4774-a345-854065d67961 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.382356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c365c1-4abb-4774-a345-854065d67961 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.382828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c365c1-4abb-4774-a345-854065d67961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.383501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee3efc-ec2d-47b9-8dea-267379581bea None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:15.403210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-38c365c1-4abb-4774-a345-854065d67961 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:15.404769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-38c365c1-4abb-4774-a345-854065d67961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:15.423784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:15.424647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.424947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.425222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.440923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:15.442401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:15.458976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:15.460038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:15.460962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:15.460962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:15.460962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:15.460962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 460/902] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:15.467255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:15.485187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:15.502715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 443/903] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:15.575398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0b810-177f-4bfd-af68-1583bb08c6ac None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:15.583395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:15.611148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:15.612008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.612439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.612861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.613680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:15.614145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:15.615990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:15.628950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 461/904] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:15.636406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c365c1-4abb-4774-a345-854065d67961 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:15.747402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c365c1-4abb-4774-a345-854065d67961 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:15.778781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:15.786960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ac13-2449-4b70-a347-d4d1475dd107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:15.787649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ac13-2449-4b70-a347-d4d1475dd107 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.787935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ac13-2449-4b70-a347-d4d1475dd107 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.788213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ac13-2449-4b70-a347-d4d1475dd107 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.796901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ac13-2449-4b70-a347-d4d1475dd107 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:15.798942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ac13-2449-4b70-a347-d4d1475dd107 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b5b5ac13-2449-4b70-a347-d4d1475dd107 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:15.813942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:15.815483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:15.816712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 444/905] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:15.817322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4d70-beba-48bd-92ce-b99163d947c8 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.817322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4d70-beba-48bd-92ce-b99163d947c8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.817483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4d70-beba-48bd-92ce-b99163d947c8 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.818683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 445/906] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:15.823973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4d70-beba-48bd-92ce-b99163d947c8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:15.881296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:15.903365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd88a3-3298-46c2-aac5-44b4431f658f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:15.940449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb6128-ca5f-4e7c-b0fb-923d7f993ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:15.940654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4d70-beba-48bd-92ce-b99163d947c8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:15.941088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6128-ca5f-4e7c-b0fb-923d7f993ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.941372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6128-ca5f-4e7c-b0fb-923d7f993ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.941647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb6128-ca5f-4e7c-b0fb-923d7f993ee3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.942360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb6128-ca5f-4e7c-b0fb-923d7f993ee3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:15.943918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb6128-ca5f-4e7c-b0fb-923d7f993ee3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c6bb6128-ca5f-4e7c-b0fb-923d7f993ee3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:15.953198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:15.954167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:15.955071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 462/907] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 24 06:31:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:15.955950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc17d-beca-447c-b257-dcb3d6de1ef5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.955950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc17d-beca-447c-b257-dcb3d6de1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.956643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc17d-beca-447c-b257-dcb3d6de1ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.974091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-87adc17d-beca-447c-b257-dcb3d6de1ef5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:15.975327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-87adc17d-beca-447c-b257-dcb3d6de1ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:15.984176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:15.985151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:15.985615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:15.986205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:15.987050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:15.987535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:15.989337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:15.999222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:16.004635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 446/908] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:16.005465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.050836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 463/909] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:16.057914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc17d-beca-447c-b257-dcb3d6de1ef5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.115944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704e083-97af-4f54-9126-ccd1aa856765 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.127543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:16.152212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.152939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.153228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.153504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.154267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:16.154695 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.157367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.163734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 447/910] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:16.170816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.185074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc17d-beca-447c-b257-dcb3d6de1ef5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.225926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.226515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.226849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.227131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.227915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] RBAC: Adding query filter params (role.id=b0feaa31560941bca79f97627f721c71, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f2837a414b814cf7915592c7f67cbd01) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:16.228373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.229920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.237729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 464/911] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/role_assignments?user.id=f2837a414b814cf7915592c7f67cbd01&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=b0feaa31560941bca79f97627f721c71 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:16.299210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18abb-6b14-44fd-a1d3-d1c28139c3cd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.336716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.340583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.341416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.341703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.341975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.342631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:16.343164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.344606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.355295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 448/912] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:16.357375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.370494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:16.416636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:16.470825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03a09-759e-4752-b0e3-e6d7fd3ea5b8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.486814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f1946-3c09-4ea7-941c-45d051d6b5b5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.517728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.518806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.519508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.519796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.520676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.522145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.524399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.525180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.525528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.525871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.526650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] RBAC: Adding query filter params (interface=public, service_id=e8b220bf56244537b579ce5209ad2bd0, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:16.527043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.528500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:16.531616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:16.532799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:16.533976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 465/913] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:16.539155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.540693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 449/914] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/endpoints?service_id=e8b220bf56244537b579ce5209ad2bd0&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:16.548241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.639091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:16.652877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c33c-08f2-4c8e-a2c6-5e7055e2c8e9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.658332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025650e2-1613-462a-a184-b9f7b1f9678c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.660189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:16.692943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.693682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.694026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.694377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.695095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:16.695497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.696959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.698489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5bf53-dc13-4e21-a700-a9d103fa1812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.699476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5bf53-dc13-4e21-a700-a9d103fa1812 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.699913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5bf53-dc13-4e21-a700-a9d103fa1812 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.700336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5bf53-dc13-4e21-a700-a9d103fa1812 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.701154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5bf53-dc13-4e21-a700-a9d103fa1812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.702625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 466/915] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:16.703634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5bf53-dc13-4e21-a700-a9d103fa1812 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.708545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.719414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 450/916] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:31:16.735365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012652cd-97e0-4e2c-b9ee-1e70ab5f73f9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.735920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012652cd-97e0-4e2c-b9ee-1e70ab5f73f9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.736383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012652cd-97e0-4e2c-b9ee-1e70ab5f73f9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.737928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 451/917] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:16.746727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247020-64b8-4fb1-a657-bfb834749b44 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.747075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247020-64b8-4fb1-a657-bfb834749b44 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.748232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247020-64b8-4fb1-a657-bfb834749b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.770055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-df247020-64b8-4fb1-a657-bfb834749b44 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:16.771127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-df247020-64b8-4fb1-a657-bfb834749b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:16.836858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e83ae5-05a0-4e53-8933-1c04f2dbc5a9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:16.854279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 452/918] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:16.881435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:16.882420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:16.882956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:16.883536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:16.884395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:16.884911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:16.886499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:16.892308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 467/919] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Aug 24 06:31:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:16.897693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df247020-64b8-4fb1-a657-bfb834749b44 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:16.995710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:17.000847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df247020-64b8-4fb1-a657-bfb834749b44 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:17.036207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:17.036819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:17.037100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:17.037394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:17.038055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:17.039463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:17.063239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 453/920] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:31:17.126953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:17.150121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:17.242922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065984a2-65eb-4178-86e0-0e6632310d48 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:17.286167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:17.286789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:17.287097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:17.287392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:17.287996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:17.289371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:17.299227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:17.300323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:17.301176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 468/921] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:17.309003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:17.421356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80ec48-d127-4012-a789-d24ed42b4084 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:17.457323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:17.458244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:17.458741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:17.459236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:17.460072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:17.460557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:17.462167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:17.467788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 454/922] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:17.473923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:17.555054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:17.579756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687922-f30d-45f9-94c8-d3618fee3226 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:17.614422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:17.615409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:17.615890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:17.616352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:17.617176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:17.617680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:17.619307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:17.625842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 469/923] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:17.632207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:17.653071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:17.745706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318696f5-2492-47dd-a2ce-cc689a45d475 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:17.781747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca08fbf-b506-425f-94dd-f82a2d1ede16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:17.783271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca08fbf-b506-425f-94dd-f82a2d1ede16 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:17.783759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca08fbf-b506-425f-94dd-f82a2d1ede16 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:17.784230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca08fbf-b506-425f-94dd-f82a2d1ede16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:17.785136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca08fbf-b506-425f-94dd-f82a2d1ede16 None admin] RBAC: Adding query filter params (interface=public, service_id=70dcf2f97cb7463b875c56154918b629, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:17.785628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca08fbf-b506-425f-94dd-f82a2d1ede16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:17.787221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca08fbf-b506-425f-94dd-f82a2d1ede16 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:17.793735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 455/924] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:31:17 2022] GET /identity/v3/endpoints?service_id=70dcf2f97cb7463b875c56154918b629&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:17.799846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:17.906240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48a873-cf34-4977-aa3f-459f8b02d888 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:17.940928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bf3b6-2d2b-4aef-bacf-8aa7b81c1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:17.941935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bf3b6-2d2b-4aef-bacf-8aa7b81c1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:17.942408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bf3b6-2d2b-4aef-bacf-8aa7b81c1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:17.942879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bf3b6-2d2b-4aef-bacf-8aa7b81c1cb5 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:17.943695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bf3b6-2d2b-4aef-bacf-8aa7b81c1cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:17.945257 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bf3b6-2d2b-4aef-bacf-8aa7b81c1cb5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:17.951908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 470/925] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:31:17 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:31:18.253031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09b10d-00b0-4fd9-a5c8-0606fa9a9c09 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:18.253329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09b10d-00b0-4fd9-a5c8-0606fa9a9c09 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:18.253585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09b10d-00b0-4fd9-a5c8-0606fa9a9c09 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:18.255226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 456/926] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:18.260422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505d53f-f59a-4e66-82f9-f1b12c70545f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:18.260968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505d53f-f59a-4e66-82f9-f1b12c70545f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:18.260968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505d53f-f59a-4e66-82f9-f1b12c70545f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:18.268843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:18.282302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6505d53f-f59a-4e66-82f9-f1b12c70545f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:18.283893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6505d53f-f59a-4e66-82f9-f1b12c70545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:18.359470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 471/927] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:18.628874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09b10d-00b0-4fd9-a5c8-0606fa9a9c09 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:18.725937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:18.730000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae82c6-2690-49e2-9074-785655c7ffcb None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:18.730360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae82c6-2690-49e2-9074-785655c7ffcb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:18.730854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae82c6-2690-49e2-9074-785655c7ffcb None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:18.733962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 472/928] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:18.740607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c7c9b-abd1-4a60-9043-f068fc2ddb93 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:18.740892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c7c9b-abd1-4a60-9043-f068fc2ddb93 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:18.741243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c7c9b-abd1-4a60-9043-f068fc2ddb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:18.760549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-564c7c9b-abd1-4a60-9043-f068fc2ddb93 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:18.761736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-564c7c9b-abd1-4a60-9043-f068fc2ddb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:18.766828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09b10d-00b0-4fd9-a5c8-0606fa9a9c09 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:18.809129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:18.810699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:18.811213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:18.811961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:18.834888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:18.840879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:18.841881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:18.842904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:18.844027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 473/929] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:18.844104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 457/930] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 06:31:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:18.849504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c7c9b-abd1-4a60-9043-f068fc2ddb93 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:18.914111 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:18.953569 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c7c9b-abd1-4a60-9043-f068fc2ddb93 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:18.987798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cd18b-3ce7-49ee-b75b-6f9577a406ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:18.988419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd18b-3ce7-49ee-b75b-6f9577a406ce None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:18.988704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd18b-3ce7-49ee-b75b-6f9577a406ce None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:18.989033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd18b-3ce7-49ee-b75b-6f9577a406ce None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:18.989868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cd18b-3ce7-49ee-b75b-6f9577a406ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:18.990180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cd18b-3ce7-49ee-b75b-6f9577a406ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:18.991608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cd18b-3ce7-49ee-b75b-6f9577a406ce None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.000476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 474/931] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 06:31:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:19.008314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.119954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5921d9-3853-4218-9743-e0f6cca425d9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.156887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.157533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.157818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.158096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.167255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:19.168739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e409cd3-2719-49dc-8ace-4a22dc599f1b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.169027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e409cd3-2719-49dc-8ace-4a22dc599f1b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.169304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e409cd3-2719-49dc-8ace-4a22dc599f1b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.170901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 475/932] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:19.176746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.177003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95c3c-41b0-471f-8e04-884390ebe6e3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.177249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95c3c-41b0-471f-8e04-884390ebe6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.177527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95c3c-41b0-471f-8e04-884390ebe6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.180105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:19.195368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:19.196428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:19.197340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:19.197340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:19.197340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:19.197340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 458/933] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:19.197580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.198119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-16a95c3c-41b0-471f-8e04-884390ebe6e3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:19.199304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-16a95c3c-41b0-471f-8e04-884390ebe6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:19.279532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 476/934] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:19.279532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95c3c-41b0-471f-8e04-884390ebe6e3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.327342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cb394-47eb-4521-9680-87a74d201bce None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.362400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:19.363454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.364163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.364515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.364859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.365623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.367118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:19.376742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:19.377747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:19.378664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 459/935] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:19.387333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.420420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95c3c-41b0-471f-8e04-884390ebe6e3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.449615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:19.462386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.463345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.463789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.464221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.464877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:19.465181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.466688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.480575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 477/936] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:19.487472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.504077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3ae22-d2a4-4d77-8d66-a1f567c4c125 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.542535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.543651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.544139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.544611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.545565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:19.546160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.547785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.558416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 460/937] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:19.559797 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.595482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c587b86-80f7-422d-b018-e32a6286b679 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.613540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:19.636574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.637426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.637747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.638034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.646141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.647645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:19.659126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:19.660359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:19.661576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.665241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 478/938] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:19.666161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.674732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec1350-bb02-42cd-8486-a77552671be2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.698016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:19.714977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.715626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.715937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.716181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.716811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:19.717111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.718602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.729253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 461/939] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:19.729253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.770819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd096961-767e-454b-8573-1f1733516e74 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.807426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.808044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.808322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.808590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.809229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.810940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.813853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING keystone.server.flask.common [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 06:31:19.835131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.sql.core [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 06:31:19.835131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 06:31:19.835131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [parameters: {'id': '2cc08c36ba924e49bd5af3811c3eb2ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 06:31:19.835131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81769) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 06:31:19.839020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baaa50-ff58-444c-985d-c3da7aca7616 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2cc08c36ba924e49bd5af3811c3eb2ad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2cc08c36ba924e49bd5af3811c3eb2ad. Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:31:19.851936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 06:31:19.852854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application yield resource Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise self.value Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application session.commit() Aug 24 06:31:19.853827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.flush() Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application compat.raise_( Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 06:31:19.854763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 06:31:19.856562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:31:19.857352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [parameters: {'id': '2cc08c36ba924e49bd5af3811c3eb2ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.859340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:19.862048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2cc08c36ba924e49bd5af3811c3eb2ad. Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 479/940] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Aug 24 06:31:19 2022] POST /identity/v3/projects => generated 262 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:19.863178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.885958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:19.886574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:19.886860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:19.887138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:19.887913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:19.888223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:19.889693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:19.894384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:19.908244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 462/941] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:19.913750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:19.960626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:19.968860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021150-bb71-4076-8d01-b37a2ac1fd5e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.006468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.007085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.007447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.007725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.008397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.009991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.021937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e0c54-4e47-4b77-8c58-c308d070ee4b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.040655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 480/942] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:20.041490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.071585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fcc11-d5d4-4984-ab0f-5ccf391278f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.072561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcc11-d5d4-4984-ab0f-5ccf391278f2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.072847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcc11-d5d4-4984-ab0f-5ccf391278f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.073119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcc11-d5d4-4984-ab0f-5ccf391278f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.081177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fcc11-d5d4-4984-ab0f-5ccf391278f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.082651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fcc11-d5d4-4984-ab0f-5ccf391278f2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-cf0fcc11-d5d4-4984-ab0f-5ccf391278f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:20.094913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:20.096169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:20.097130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 463/943] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:20.098026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2286-4bff-4869-87b2-9edcecb868b8 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.098305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2286-4bff-4869-87b2-9edcecb868b8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.098580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2286-4bff-4869-87b2-9edcecb868b8 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.099952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 464/944] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:20.102454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2286-4bff-4869-87b2-9edcecb868b8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.162423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:20.179277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3111a2-ebcf-4962-ae19-c45abb7b0408 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.220436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:20.227267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2286-4bff-4869-87b2-9edcecb868b8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.227487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e110cc6-a14e-4d22-8f86-288ea4948de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.228134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e110cc6-a14e-4d22-8f86-288ea4948de2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.228415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e110cc6-a14e-4d22-8f86-288ea4948de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.228779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e110cc6-a14e-4d22-8f86-288ea4948de2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.241758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e110cc6-a14e-4d22-8f86-288ea4948de2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.243273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e110cc6-a14e-4d22-8f86-288ea4948de2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5e110cc6-a14e-4d22-8f86-288ea4948de2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:20.258398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:20.259600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:20.260728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:20.260728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:20.260728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:20.260728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 481/945] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:20.261284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b97d-8ac5-42ef-aac6-da82fac37abc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.261572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b97d-8ac5-42ef-aac6-da82fac37abc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.261851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b97d-8ac5-42ef-aac6-da82fac37abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.268773 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.269256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.269538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.269815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.270483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:20.270787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.272241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.281204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d060b97d-8ac5-42ef-aac6-da82fac37abc None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:20.282745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d060b97d-8ac5-42ef-aac6-da82fac37abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:20.292840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 465/946] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:20.294498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.382156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 482/947] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:20.419484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c15561-3daf-4cc5-a0f7-e2a64a645613 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.458953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.459606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.459886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.460160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.460809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:20.461113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.462524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.475105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 466/948] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:20.480725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b97d-8ac5-42ef-aac6-da82fac37abc None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.556601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:20.590611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b97d-8ac5-42ef-aac6-da82fac37abc None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.626385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.627039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.627358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.627639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.634890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.636388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:20.647775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:20.648667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:20.649567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 483/949] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:20.653006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.657543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.675224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:20.779684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78755da0-7f29-44b6-8471-07d19f12eb0e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.782876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611c053-afd7-499b-a17a-07173fcbc9fb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.814161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:20.821870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.822772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.823489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.824066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.824483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:20.825122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:20.825400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:20.825671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:20.825859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:20.826258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.826633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:20.827840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.828838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:20.838438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:20.840140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:20.841042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 467/950] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:20.847852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.848978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 484/951] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:20.855103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:20.957254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:20.974182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffee209-a074-4788-8cdd-0481740e06e1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:20.980131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9444d-2e06-4345-a7f1-36d38ffd6fc2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:21.016796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:21.017944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:21.018616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:21.019182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:21.019778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:21.020444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:21.020556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:21.020728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:21.021047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f2837a414b814cf7915592c7f67cbd01/roles/b0feaa31560941bca79f97627f721c71` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:21.021616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:21.023265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=f2837a414b814cf7915592c7f67cbd01, role_id=b0feaa31560941bca79f97627f721c71)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:21.023649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:21.024684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:21.034157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 468/952] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:21.036192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:21.059486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 485/953] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:31:20 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/f2837a414b814cf7915592c7f67cbd01/roles/b0feaa31560941bca79f97627f721c71 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:31:21.150285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a22a8-8678-4599-aed5-6720408f25dd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:21.192919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e28f51-939a-4b14-826a-e2feb94db8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:21.193933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28f51-939a-4b14-826a-e2feb94db8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:21.194338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28f51-939a-4b14-826a-e2feb94db8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:21.194612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e28f51-939a-4b14-826a-e2feb94db8a4 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:21.195484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28f51-939a-4b14-826a-e2feb94db8a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:21.195827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28f51-939a-4b14-826a-e2feb94db8a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:21.197768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e28f51-939a-4b14-826a-e2feb94db8a4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:21.203884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 469/954] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:21.210162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:21.272768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:21.328905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1a7c-01ea-4cf7-af18-e8ececb373e6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:21.369236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436286a-1c01-43f4-8956-996f1f677a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:21.370146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436286a-1c01-43f4-8956-996f1f677a38 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:21.370581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436286a-1c01-43f4-8956-996f1f677a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:21.371003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436286a-1c01-43f4-8956-996f1f677a38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:21.371849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436286a-1c01-43f4-8956-996f1f677a38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:21.373402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436286a-1c01-43f4-8956-996f1f677a38 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:21.398441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 486/955] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:22.004836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96004f-f0a7-4f43-8a3b-3bb38da573a3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.005388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96004f-f0a7-4f43-8a3b-3bb38da573a3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.005622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96004f-f0a7-4f43-8a3b-3bb38da573a3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.007366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 470/956] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:22.012659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd68d1-4004-47e6-976e-737ac3f04bd1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.013281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd68d1-4004-47e6-976e-737ac3f04bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.013653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd68d1-4004-47e6-976e-737ac3f04bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.032841 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bdfd68d1-4004-47e6-976e-737ac3f04bd1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:22.034046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bdfd68d1-4004-47e6-976e-737ac3f04bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:22.114055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 487/957] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:22.380979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd68d1-4004-47e6-976e-737ac3f04bd1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:22.381607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb21518-ba44-4e5a-8240-3d0c5c4ab053 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.381607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb21518-ba44-4e5a-8240-3d0c5c4ab053 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.381782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb21518-ba44-4e5a-8240-3d0c5c4ab053 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.383314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 471/958] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:22.393380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa6ae9-6206-484d-a9f1-1549e6873269 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.393671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa6ae9-6206-484d-a9f1-1549e6873269 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.393951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa6ae9-6206-484d-a9f1-1549e6873269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.412480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-66aa6ae9-6206-484d-a9f1-1549e6873269 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:22.413250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:22.414922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-66aa6ae9-6206-484d-a9f1-1549e6873269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:22.418421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:22.485552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd68d1-4004-47e6-976e-737ac3f04bd1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:22.503387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 472/959] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:22.521044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:22.521703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.521983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.522255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.522861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:22.524325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:22.534156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:22.535384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:22.536318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 488/960] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:22.543229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa6ae9-6206-484d-a9f1-1549e6873269 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:22.644557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa6ae9-6206-484d-a9f1-1549e6873269 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:22.679400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:22.680013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.680292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.680565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.681203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:22.681509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:22.682865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:22.687997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 473/961] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:22.694025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:22.791516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:22.792781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:22.797319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:22.805400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86c5f8-45fc-4f62-8064-4a7e23d279a4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:22.844350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:22.845379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.845956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.846428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.847300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:22.847805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:22.849595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:22.855446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 489/962] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:22.913224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c98422-75f5-4ac6-bf16-4af022657d54 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:22.952417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:22.953540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:22.954056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:22.954528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:22.955430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:22.957125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:22.960103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING keystone.server.flask.common [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 06:31:22.974909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.sql.core [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 06:31:22.974909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 06:31:22.974909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [parameters: {'user_id': '9c5dfbd5bbd24bbe87329aa029e953c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 06:31:22.974909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81768) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4f0fb717-19f6-4e7f-ba3d-3ebd310af460 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result.read() Aug 24 06:31:22.982995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:31:22.984090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application yield resource Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise self.value Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application session.commit() Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 06:31:22.985079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.session.flush() Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application compat.raise_( Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 06:31:22.985985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:31:22.986922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application result.read() Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:31:22.987833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [parameters: {'user_id': '9c5dfbd5bbd24bbe87329aa029e953c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:22.988684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 06:31:22.989511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 06:31:22.993003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 06:31:22.993003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 06:31:22.993003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 06:31:22.993003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 06:31:22.993003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 06:31:22.993003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:22.999364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 474/963] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Aug 24 06:31:22 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:23.002726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:23.089616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1008f3-058e-41ce-bc04-51793c054eea None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.090195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1008f3-058e-41ce-bc04-51793c054eea None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.090355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1008f3-058e-41ce-bc04-51793c054eea None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.096111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 475/964] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:23.101743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446aed-a7fe-45c2-8707-854f6c3fe402 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.102289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446aed-a7fe-45c2-8707-854f6c3fe402 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.102803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c446aed-a7fe-45c2-8707-854f6c3fe402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.116069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88788d24-3ed1-498a-872e-f7690fb21379 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:23.122662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5c446aed-a7fe-45c2-8707-854f6c3fe402 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:23.127238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5c446aed-a7fe-45c2-8707-854f6c3fe402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:23.161960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:23.162821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.163223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.163654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.174049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:23.175533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:23.187416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:23.188456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:23.189477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 490/965] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:31:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:23.195936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:23.197321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:23.220050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 476/966] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:23.227104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:23.309243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e02fb-1832-47da-88c1-ee066650a5af None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:23.344704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0201c-3235-47ae-8c16-201fd529f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:23.345563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0201c-3235-47ae-8c16-201fd529f4de None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.346102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0201c-3235-47ae-8c16-201fd529f4de None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.346544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0201c-3235-47ae-8c16-201fd529f4de None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.347419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0201c-3235-47ae-8c16-201fd529f4de None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:23.347868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0201c-3235-47ae-8c16-201fd529f4de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:23.349469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0201c-3235-47ae-8c16-201fd529f4de None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:23.387132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 491/967] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Aug 24 06:31:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:23.388610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3226de-14c6-4f8c-bbb3-d1daf0d2d82c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.388907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3226de-14c6-4f8c-bbb3-d1daf0d2d82c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.389188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3226de-14c6-4f8c-bbb3-d1daf0d2d82c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.390787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 477/968] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:23.397311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7277bb-cba3-4c29-8efb-6489937b1dcb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.397976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7277bb-cba3-4c29-8efb-6489937b1dcb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.398459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7277bb-cba3-4c29-8efb-6489937b1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.419521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ca7277bb-cba3-4c29-8efb-6489937b1dcb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:23.421242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ca7277bb-cba3-4c29-8efb-6489937b1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:23.497033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3226de-14c6-4f8c-bbb3-d1daf0d2d82c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:23.507221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 492/969] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:23.616860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3226de-14c6-4f8c-bbb3-d1daf0d2d82c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:23.654499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:23.655547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.655691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.656030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.667023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:23.672101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:23.673255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:23.674294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:23.675425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 478/970] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 06:31:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:23.681063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7277bb-cba3-4c29-8efb-6489937b1dcb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:23.692513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:23.782712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7277bb-cba3-4c29-8efb-6489937b1dcb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:23.801698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:23.819802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:23.820419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.820704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.820981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.821656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:23.821966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:23.823379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:23.831989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 493/971] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 06:31:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:23.837434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:23.865613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:23.911979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886bacf-5388-4eee-8f1e-b711ed72d9a7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:23.912235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:23.948223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06487c-1221-44a9-95ab-d7b3f47b57ba None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:23.949949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:23.951416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.951701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.951974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.953301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:23.954854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:23.964329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:23.965387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:23.969252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 479/972] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:23.972127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:23.985714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:23.986448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:23.986789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:23.987069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:23.999321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.000786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:24.015633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:24.016744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:24.017665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:24.017665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:24.017665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:24.021674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 494/973] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:24.022454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.087740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c8e5b-30cb-40cd-9031-47bb599df055 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.111957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:24.128018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.128637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.128920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.129199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.129862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:24.130168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.130387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2f75b-aef4-4f11-8046-b7535cff626f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.131573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.137664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 480/974] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:24.143563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.157371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:24.172326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.172949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.173234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.173521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.174157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:24.174461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.175872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.188290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 495/975] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:24.194351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.258729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfed458-3efe-46b0-9181-24f5d2ee17a1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.296569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.297335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.297543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.297930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.298593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:24.298872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.300480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.301146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7691e-598c-46a1-84fd-48e406161144 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.306270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 481/976] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:24.312034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.340989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.341643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.342046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.342328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.350560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.352172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.360290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:24.365630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:24.366648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:24.367740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 496/977] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:24.375769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.418572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6106aa-6afe-4b3e-87e1-ed88cda04915 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.423412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:24.454163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.454795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.455078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.455383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.456107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] RBAC: Adding query filter params (interface=admin, service_id=70dcf2f97cb7463b875c56154918b629, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:24.456422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.457798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.463252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 482/978] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/endpoints?service_id=70dcf2f97cb7463b875c56154918b629&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:24.469023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.495486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166b9d0-a618-49eb-b64f-526caa34450a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.532947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.533868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.534439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.535183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.536397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:24.536874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.538632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.560028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 497/979] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:24.566530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.575382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3719841-9cdd-4918-b9d0-497d1e46314f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.589413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:24.614426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be9f0b-9a5e-4682-8dd3-36a9a23fd501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.615134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be9f0b-9a5e-4682-8dd3-36a9a23fd501 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.615458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be9f0b-9a5e-4682-8dd3-36a9a23fd501 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.615755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be9f0b-9a5e-4682-8dd3-36a9a23fd501 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.616366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be9f0b-9a5e-4682-8dd3-36a9a23fd501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.617738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be9f0b-9a5e-4682-8dd3-36a9a23fd501 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.631629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 483/980] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:31:24.679818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:24.683071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7fbc-36d3-45f8-a784-6162b5db5996 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:24.725109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:24.725648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.725930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.726196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.805673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:24.807187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:24.904737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:24.910749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70869263-1d9c-4995-9889-fcae3a6b3eba None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.911040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70869263-1d9c-4995-9889-fcae3a6b3eba None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.915231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 498/981] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:24.915373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70869263-1d9c-4995-9889-fcae3a6b3eba None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.915373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 484/982] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:24.915563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:24.918950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4fee3-23aa-439c-a0dc-af039847533d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:24.919270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4fee3-23aa-439c-a0dc-af039847533d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:24.919561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4fee3-23aa-439c-a0dc-af039847533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:24.939833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b4f4fee3-23aa-439c-a0dc-af039847533d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:24.941407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b4f4fee3-23aa-439c-a0dc-af039847533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:25.010987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 485/983] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:25.019848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f94a84-f02e-4b4a-9f23-ea2cd1cce9fd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:25.055325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:25.055965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:25.056246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:25.056527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:25.067689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:25.069163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:25.089449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:25.092363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:25.098768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:25.098768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:25.098768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:25.098768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 499/984] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:25.100499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4fee3-23aa-439c-a0dc-af039847533d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:25.205924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4fee3-23aa-439c-a0dc-af039847533d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:25.219764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:25.248268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:25.248938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:25.249405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:25.250122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:25.250780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:25.251088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:25.252534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:25.264855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 486/985] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:25.270936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:25.281332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:25.329353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:25.380926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e10a6-d2a8-416a-b8ab-3f7014253171 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:25.386637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac339e33-413d-4a00-afdc-edddc9e1cd39 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:25.417041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:25.417677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:25.417961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:25.418233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:25.419918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:25.420591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:25.420959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:25.421255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:25.421898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:25.423294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:25.425702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:25.427140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:25.439057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:25.440410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:25.441378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:25.445751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 500/986] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:25.446724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:25.447514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 487/987] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:31:25 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:25.569856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c68f26-333d-461e-a623-e5f3f26abcab None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:25.595555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:25.629140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec8d50-98a1-40c1-aa16-3ab613a0d3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:25.629765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8d50-98a1-40c1-aa16-3ab613a0d3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:25.630046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8d50-98a1-40c1-aa16-3ab613a0d3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:25.630322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec8d50-98a1-40c1-aa16-3ab613a0d3c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:25.631061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec8d50-98a1-40c1-aa16-3ab613a0d3c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:25.631403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec8d50-98a1-40c1-aa16-3ab613a0d3c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:25.632887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec8d50-98a1-40c1-aa16-3ab613a0d3c5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:25.647137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 501/988] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:25.653153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:25.682938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:25.763738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582221c7-c570-404c-a877-5c8fd647e81f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:25.797851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce89881-bf4d-45af-8f8a-513e64e57c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:25.798470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce89881-bf4d-45af-8f8a-513e64e57c40 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:25.798758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce89881-bf4d-45af-8f8a-513e64e57c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:25.799034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce89881-bf4d-45af-8f8a-513e64e57c40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:25.799813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce89881-bf4d-45af-8f8a-513e64e57c40 None admin] RBAC: Adding query filter params (role.id=b0feaa31560941bca79f97627f721c71, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=f2837a414b814cf7915592c7f67cbd01) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:25.800120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce89881-bf4d-45af-8f8a-513e64e57c40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:25.801561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce89881-bf4d-45af-8f8a-513e64e57c40 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:25.815354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 488/989] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:25 2022] GET /identity/v3/role_assignments?user.id=f2837a414b814cf7915592c7f67cbd01&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=b0feaa31560941bca79f97627f721c71 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:26.546639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d788a5c-c415-4d4b-8cdd-8413dc767f54 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:26.547280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d788a5c-c415-4d4b-8cdd-8413dc767f54 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:26.547280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d788a5c-c415-4d4b-8cdd-8413dc767f54 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:26.548826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 502/990] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:26.554304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f800e9c-77df-4337-88da-3290dbcae3ca None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:26.554738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f800e9c-77df-4337-88da-3290dbcae3ca None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:26.554915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f800e9c-77df-4337-88da-3290dbcae3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:26.573818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9f800e9c-77df-4337-88da-3290dbcae3ca None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:26.575015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9f800e9c-77df-4337-88da-3290dbcae3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:26.646114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 489/991] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:26.907504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d788a5c-c415-4d4b-8cdd-8413dc767f54 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:27.015797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d788a5c-c415-4d4b-8cdd-8413dc767f54 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:27.049948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:27.050591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.050877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.051152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.051780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:27.053168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:27.062272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:27.063351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:27.068356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 503/992] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Aug 24 06:31:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:27.070336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f800e9c-77df-4337-88da-3290dbcae3ca None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:27.092321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:27.173145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f800e9c-77df-4337-88da-3290dbcae3ca None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:27.207417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8765d-2364-4f94-a146-91f108534e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:27.208031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8765d-2364-4f94-a146-91f108534e1d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.208318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8765d-2364-4f94-a146-91f108534e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.208578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8765d-2364-4f94-a146-91f108534e1d None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.212145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8765d-2364-4f94-a146-91f108534e1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:27.212145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8765d-2364-4f94-a146-91f108534e1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:27.212145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8765d-2364-4f94-a146-91f108534e1d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:27.220656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 490/993] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:27.226608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:27.261304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:27.335087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ffc76-e81e-43c9-9e62-36526dc43636 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:27.364803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec0f64-f207-4bb9-a7f5-56cf9cb65ccb None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.365086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec0f64-f207-4bb9-a7f5-56cf9cb65ccb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.365365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec0f64-f207-4bb9-a7f5-56cf9cb65ccb None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.366731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 491/994] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:27.371653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72207a-7743-4ba9-ae7f-d3c31ec2a74b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.371938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72207a-7743-4ba9-ae7f-d3c31ec2a74b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.372211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72207a-7743-4ba9-ae7f-d3c31ec2a74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.373279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:27.373977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.374285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.374895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.375177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:27.375530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:27.376948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:27.382175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 504/995] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Aug 24 06:31:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:27.388315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:27.393977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4e72207a-7743-4ba9-ae7f-d3c31ec2a74b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:27.395168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4e72207a-7743-4ba9-ae7f-d3c31ec2a74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:27.474440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 492/996] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:27.497139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:27.504581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24913bd7-8490-4a64-ac26-f7b3002d43ac None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:27.539343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5d930-ad35-40f0-ae5a-2b9d79eb81bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:27.539955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d930-ad35-40f0-ae5a-2b9d79eb81bf None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.540235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d930-ad35-40f0-ae5a-2b9d79eb81bf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.540508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5d930-ad35-40f0-ae5a-2b9d79eb81bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.541223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5d930-ad35-40f0-ae5a-2b9d79eb81bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:27.542693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5d930-ad35-40f0-ae5a-2b9d79eb81bf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:27.567304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 505/997] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:31:27.744023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72207a-7743-4ba9-ae7f-d3c31ec2a74b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:27.759766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:27.761145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28745f-f4ee-4218-abd9-6271c3e45376 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.761623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28745f-f4ee-4218-abd9-6271c3e45376 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.761927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28745f-f4ee-4218-abd9-6271c3e45376 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.763632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 506/998] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:27.770390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf4303-ab16-455b-83fe-378403e5a584 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.771060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf4303-ab16-455b-83fe-378403e5a584 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.771564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf4303-ab16-455b-83fe-378403e5a584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.791455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-23bf4303-ab16-455b-83fe-378403e5a584 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:27.792665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-23bf4303-ab16-455b-83fe-378403e5a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:27.859414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72207a-7743-4ba9-ae7f-d3c31ec2a74b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:27.878141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 507/999] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:27 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:27.900574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:27.901248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:27.901529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:27.903710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:27.904353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:27.905855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:27.915105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:27.916164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:27.917033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:27.918420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 493/1000] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:27.922166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf4303-ab16-455b-83fe-378403e5a584 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:28.007976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:28.029549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf4303-ab16-455b-83fe-378403e5a584 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:28.063627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:28.064238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:28.064516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:28.064788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:28.065782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:28.066083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:28.067479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:28.072219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 508/1001] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:28.078134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:28.146602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:28.161219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:28.184372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2cec0-41d1-4afc-8b6c-f6d35ddb1e39 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:28.230652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:28.231359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:28.231655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:28.231855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:28.232499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:28.232807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:28.234267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:28.239583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 494/1002] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:28.245249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:28.277700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841efeda-2f1f-42b5-b4bc-d0ff646c8fb2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:28.313615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:28.314252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:28.314531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:28.314806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] PATH_INFO: `/v3/services/image` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:28.315813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:28.317195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:28.325700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:28.326908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:28.328583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 509/1003] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 24 06:31:28 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:28.333004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:28.345480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:28.357485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed06b0-f75e-4466-abc8-1bb7e8ca6c12 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:28.394266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:28.394880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:28.395169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:28.395478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:28.396190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] RBAC: Adding query filter params (interface=public, service_id=3e4fce66cc1346fda5f33bf4d6438dce, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:28.396510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:28.397892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:28.403223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 495/1004] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:31:28 2022] GET /identity/v3/endpoints?service_id=3e4fce66cc1346fda5f33bf4d6438dce&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:28.409400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:28.419404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:28.447640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44096ec4-4649-49e0-ad8a-e26d6b412586 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:28.482818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:28.484173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:28.484521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:28.484865 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:28.485583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:28.485957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:28.487434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:28.492478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 510/1005] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:28.498178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:28.522211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b8300-9d06-4b7b-b7b3-796eac12db8d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:28.559717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3902b-2df8-414a-bf62-583552aa38fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:28.560249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3902b-2df8-414a-bf62-583552aa38fc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:28.560533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3902b-2df8-414a-bf62-583552aa38fc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:28.560807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3902b-2df8-414a-bf62-583552aa38fc None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:28.561551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3902b-2df8-414a-bf62-583552aa38fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:28.563079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3902b-2df8-414a-bf62-583552aa38fc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:28.570058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 496/1006] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:31:28 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:31:28.576685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:28.609345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f084858-9331-4f68-97d6-962c2b0506a7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:28.651557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9969d0-8a44-46db-85a1-ae6d4be7f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:28.652190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9969d0-8a44-46db-85a1-ae6d4be7f627 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:28.652471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9969d0-8a44-46db-85a1-ae6d4be7f627 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:28.652738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9969d0-8a44-46db-85a1-ae6d4be7f627 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:28.653394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9969d0-8a44-46db-85a1-ae6d4be7f627 None admin] RBAC: Adding query filter params (type=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:28.653688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9969d0-8a44-46db-85a1-ae6d4be7f627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:28.655145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9969d0-8a44-46db-85a1-ae6d4be7f627 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:28.661331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 511/1007] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:29.535619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8b58-eda7-45cb-ad4e-b19b9f1f5bdf None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:29.536189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8b58-eda7-45cb-ad4e-b19b9f1f5bdf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:29.536189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8b58-eda7-45cb-ad4e-b19b9f1f5bdf None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:29.537836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 497/1008] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:29.543051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389b71e-3e0b-4e5b-af0f-c8a0fc8de33d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:29.543390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389b71e-3e0b-4e5b-af0f-c8a0fc8de33d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:29.543671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389b71e-3e0b-4e5b-af0f-c8a0fc8de33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:29.563604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3389b71e-3e0b-4e5b-af0f-c8a0fc8de33d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:29.564822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3389b71e-3e0b-4e5b-af0f-c8a0fc8de33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:29.639949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 512/1009] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:29.903987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8b58-eda7-45cb-ad4e-b19b9f1f5bdf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:29.981460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:30.015938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8b58-eda7-45cb-ad4e-b19b9f1f5bdf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:30.052161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adb078-5051-412c-a7a9-37ba2ac10283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:30.052705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adb078-5051-412c-a7a9-37ba2ac10283 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:30.052983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adb078-5051-412c-a7a9-37ba2ac10283 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:30.053257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adb078-5051-412c-a7a9-37ba2ac10283 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:30.054389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adb078-5051-412c-a7a9-37ba2ac10283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:30.055879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adb078-5051-412c-a7a9-37ba2ac10283 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:30.057673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 498/1010] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:30.063975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389b71e-3e0b-4e5b-af0f-c8a0fc8de33d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:30.135308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:30.170646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389b71e-3e0b-4e5b-af0f-c8a0fc8de33d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:30.205493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636712a5-e153-4ffb-8bf9-ef16cb4357e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:30.206108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636712a5-e153-4ffb-8bf9-ef16cb4357e0 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:30.206387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636712a5-e153-4ffb-8bf9-ef16cb4357e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:30.206654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636712a5-e153-4ffb-8bf9-ef16cb4357e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:30.207337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636712a5-e153-4ffb-8bf9-ef16cb4357e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:30.211880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636712a5-e153-4ffb-8bf9-ef16cb4357e0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:30.247224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 513/1011] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:31:30 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:30.526173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15817-1cba-4382-afb8-1660dc05303e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:30.526414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15817-1cba-4382-afb8-1660dc05303e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:30.526689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15817-1cba-4382-afb8-1660dc05303e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:30.528103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 499/1012] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:30.534250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b58f7a-446d-470e-aaac-73a3b4749b9a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:30.534522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b58f7a-446d-470e-aaac-73a3b4749b9a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:30.534822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b58f7a-446d-470e-aaac-73a3b4749b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:30.554021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-62b58f7a-446d-470e-aaac-73a3b4749b9a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:30.555235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-62b58f7a-446d-470e-aaac-73a3b4749b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:30.565881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96d65-8081-4471-9a6d-ac80bc24f943 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:30.566401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96d65-8081-4471-9a6d-ac80bc24f943 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:30.566491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96d65-8081-4471-9a6d-ac80bc24f943 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:30.568088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 500/1013] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:30.573253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd21f9-86b4-40e7-b6d4-396ded75d933 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:30.573517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd21f9-86b4-40e7-b6d4-396ded75d933 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:30.573827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd21f9-86b4-40e7-b6d4-396ded75d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:30.592173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2fcd21f9-86b4-40e7-b6d4-396ded75d933 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:30.593400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2fcd21f9-86b4-40e7-b6d4-396ded75d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:30.630598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 514/1014] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:30.665052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 501/1015] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:30.900765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b58f7a-446d-470e-aaac-73a3b4749b9a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:30.932555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd21f9-86b4-40e7-b6d4-396ded75d933 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:30.962879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:31.005954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:31.019549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b58f7a-446d-470e-aaac-73a3b4749b9a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:31.043791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd21f9-86b4-40e7-b6d4-396ded75d933 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:31.057674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:31.058307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:31.058591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:31.058864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:31.059505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:31.060895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:31.070105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:31.075392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:31.076397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 515/1016] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:31.078505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:31.087494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:31.088122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:31.088402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:31.088679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:31.089345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:31.090714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:31.111238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 502/1017] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:31:30 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:31:31.192890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4c01b-7adb-470e-b725-95aec6206a2d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:31.208713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:31.244353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:31.245015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:31.245308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:31.245584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:31.246187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:31.246488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:31.247936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:31.256845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 516/1018] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:31.262562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:31.370496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac0fd1-de65-4cf7-9cd5-0a605d1f90d2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:31.397572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:31.407017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f87755-d857-4b1c-934b-d10ea76d80e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:31.407839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f87755-d857-4b1c-934b-d10ea76d80e6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:31.408165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f87755-d857-4b1c-934b-d10ea76d80e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:31.408615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f87755-d857-4b1c-934b-d10ea76d80e6 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:31.409273 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f87755-d857-4b1c-934b-d10ea76d80e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:31.409587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f87755-d857-4b1c-934b-d10ea76d80e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:31.410978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f87755-d857-4b1c-934b-d10ea76d80e6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:31.416608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 503/1019] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Aug 24 06:31:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:31.422415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:31.529008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c7a13-f8c2-4c9b-b0e7-24801008bc53 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:31.564491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f42f0d-8184-4e4c-8e42-50ee7f8bc460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:31.565443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42f0d-8184-4e4c-8e42-50ee7f8bc460 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:31.565745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42f0d-8184-4e4c-8e42-50ee7f8bc460 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:31.566018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42f0d-8184-4e4c-8e42-50ee7f8bc460 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:31.566732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f42f0d-8184-4e4c-8e42-50ee7f8bc460 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:31.568243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f42f0d-8184-4e4c-8e42-50ee7f8bc460 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:31.589107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 517/1020] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:32.206386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4b949-89a6-4455-b2d0-d4877b016e23 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:32.223116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4b949-89a6-4455-b2d0-d4877b016e23 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:32.223116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4b949-89a6-4455-b2d0-d4877b016e23 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:32.223116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 504/1021] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:32.224455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eb327-2f2e-4ac3-a7b5-b0ddbbc60c3f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:32.224455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eb327-2f2e-4ac3-a7b5-b0ddbbc60c3f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:32.224455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eb327-2f2e-4ac3-a7b5-b0ddbbc60c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:32.238101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7e2eb327-2f2e-4ac3-a7b5-b0ddbbc60c3f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:32.239329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7e2eb327-2f2e-4ac3-a7b5-b0ddbbc60c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:32.263910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:32.317079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 518/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:32.585481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4b949-89a6-4455-b2d0-d4877b016e23 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:32.692045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4b949-89a6-4455-b2d0-d4877b016e23 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:32.728855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8d8b6-0e6f-4e02-a4f5-f858d72a56f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:32.729498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8d8b6-0e6f-4e02-a4f5-f858d72a56f9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:32.729783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8d8b6-0e6f-4e02-a4f5-f858d72a56f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:32.730059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8d8b6-0e6f-4e02-a4f5-f858d72a56f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:32.731206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8d8b6-0e6f-4e02-a4f5-f858d72a56f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:32.732679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8d8b6-0e6f-4e02-a4f5-f858d72a56f9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:32.734519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 505/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:32.740250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eb327-2f2e-4ac3-a7b5-b0ddbbc60c3f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:32.843848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eb327-2f2e-4ac3-a7b5-b0ddbbc60c3f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:32.878825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163b625-41c1-4b79-8a8b-4294269d31dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:32.879519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b625-41c1-4b79-8a8b-4294269d31dd None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:32.879805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b625-41c1-4b79-8a8b-4294269d31dd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:32.880078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b625-41c1-4b79-8a8b-4294269d31dd None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:32.880733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163b625-41c1-4b79-8a8b-4294269d31dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:32.882188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163b625-41c1-4b79-8a8b-4294269d31dd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:32.911215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 519/1024] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:31:32 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:33.054438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54150d58-4a5f-4cbc-86cc-f00ba7b09fa2 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:33.054438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54150d58-4a5f-4cbc-86cc-f00ba7b09fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:33.054438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54150d58-4a5f-4cbc-86cc-f00ba7b09fa2 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:33.054438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 506/1025] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:33.059903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094703b-5fa2-4182-9784-6d003f293117 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:33.060127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094703b-5fa2-4182-9784-6d003f293117 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:33.060392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094703b-5fa2-4182-9784-6d003f293117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:33.080074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2094703b-5fa2-4182-9784-6d003f293117 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:33.081278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2094703b-5fa2-4182-9784-6d003f293117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:33.089220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:33.152614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 520/1026] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:33.420881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54150d58-4a5f-4cbc-86cc-f00ba7b09fa2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:33.484727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:33.532361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54150d58-4a5f-4cbc-86cc-f00ba7b09fa2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:33.568426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:33.569068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:33.569350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:33.569626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] PATH_INFO: `/v3/services/image` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:33.570215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:33.571714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:33.580507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:33.581623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:33.582776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:33.586568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 507/1027] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 24 06:31:33 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:33.587038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd9a8a-c828-440d-a475-53dd33bb7f33 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:33.587609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:33.587893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd9a8a-c828-440d-a475-53dd33bb7f33 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:33.587893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd9a8a-c828-440d-a475-53dd33bb7f33 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:33.589066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 521/1028] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:33.594264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830cc45-ea70-4558-8abc-06fc2e7f2478 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:33.594539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830cc45-ea70-4558-8abc-06fc2e7f2478 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:33.594829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830cc45-ea70-4558-8abc-06fc2e7f2478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:33.612939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6830cc45-ea70-4558-8abc-06fc2e7f2478 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:33.614151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6830cc45-ea70-4558-8abc-06fc2e7f2478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:33.688515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 522/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:31:33.697630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb092679-88c2-4e1f-b650-fdc7b891f00f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:33.722257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:33.734247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:33.734853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:33.735131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:33.735435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:33.736062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] RBAC: Adding query filter params (name=image) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:33.736426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:33.737792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:33.742862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 508/1030] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:33.748864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830cc45-ea70-4558-8abc-06fc2e7f2478 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:33.825116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:33.857519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830cc45-ea70-4558-8abc-06fc2e7f2478 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:33.895832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:33.896464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:33.896749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:33.897019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:33.897666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] RBAC: Adding query filter params (type=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:33.897975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:33.899458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:33.909503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 523/1031] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:33.911120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:33.953156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.020450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292cee9-2109-47c9-90e8-b958455bd24c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.056320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.056937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.057224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.057502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.058212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] RBAC: Adding query filter params (interface=public, service_id=98350bc06afe4c90887f9e1faf2a5400, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:34.058528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:34.058719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451ad5f-f2d5-486c-be3b-a2eaa5fa8988 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.059936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:34.070528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 509/1032] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:31:33 2022] GET /identity/v3/endpoints?service_id=98350bc06afe4c90887f9e1faf2a5400&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:34.071530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.102959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.103643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.103898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.104168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.106994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:34.115978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:34.120694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:34.121645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:34.123530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 524/1033] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 06:31:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:34.128307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.132862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:34.187289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3093cc-6447-4122-b29e-1654d4cb81bc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.227639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.228280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.228551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.228832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.229433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:34.230806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:34.237684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 510/1034] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:31:34 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:31:34.251013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee99b53-071b-4131-aeb1-bd5ad7a65d77 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.292890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.293800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.294275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.294694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.295586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:34.296098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:34.298467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:34.308716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 525/1035] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 06:31:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:34.314503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.429674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c068-26d2-44ca-b753-6bc8a9947479 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.447620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:34.466950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.467551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.467990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.468427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.479926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:34.481372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:34.496435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:34.497431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:34.498742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:34.498742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:34.498742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:34.498742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 511/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:34.504192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.574526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:34.620700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ed6ae-eec4-46d6-83a1-c9ad13eb62e8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.655691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.656299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.656590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.656863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.657576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:34.657885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:34.659354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:34.676154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 526/1037] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:34.677541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.786491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a167d85-5153-423f-8aa4-be3a8bfffaa2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.820849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513f792-7a95-4c27-a047-ced163852b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:34.821501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513f792-7a95-4c27-a047-ced163852b7a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.821784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513f792-7a95-4c27-a047-ced163852b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.822059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513f792-7a95-4c27-a047-ced163852b7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.829817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513f792-7a95-4c27-a047-ced163852b7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:34.831308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513f792-7a95-4c27-a047-ced163852b7a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f513f792-7a95-4c27-a047-ced163852b7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:34.842299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:34.843893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:34.845141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:34.846218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 512/1038] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:31:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:34.849419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:34.914015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e2e47-326b-41f9-9908-d79e25d69451 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.914274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e2e47-326b-41f9-9908-d79e25d69451 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.914545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e2e47-326b-41f9-9908-d79e25d69451 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.915958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 513/1039] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:34.921162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c039f-5b02-4515-acd2-3816b0908090 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:34.921489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c039f-5b02-4515-acd2-3816b0908090 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:34.921739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c039f-5b02-4515-acd2-3816b0908090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:34.940876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-959c039f-5b02-4515-acd2-3816b0908090 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:34.942089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-959c039f-5b02-4515-acd2-3816b0908090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:34.962379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e928d4-09a5-48d9-bfd3-3759ae7a224c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:34.968493 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:34.984510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:35.004134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.004755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.005038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.005311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.006258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:35.006374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:35.007838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:35.021334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 514/1040] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:35.025789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 527/1041] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:31:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:35.031634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c039f-5b02-4515-acd2-3816b0908090 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.134140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c039f-5b02-4515-acd2-3816b0908090 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:35.169501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.170077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.170372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.170640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.197907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:35.245486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:35.246957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:35.289869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.368014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 515/1042] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:35.373401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.412944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc2965-6751-469c-bb55-3e1555f1f8e8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:35.433285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:35.458960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.460515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.460515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.461187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.474963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:35.479515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c413-f489-4d74-884c-0dfcfe370bf2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:35.481333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:35.482268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:35.483141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:35.488699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 528/1043] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:35.490397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.527557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.528197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.528485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.528763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.541011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:35.542242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:35.542503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:35.560080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:35.561125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:35.567557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:35.567557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:35.567557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:35.567557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 516/1044] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:35.572486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.638868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d3f7b-b0ee-46a0-b25f-3edb2fe16157 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:35.682605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.683681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.684169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.684713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.685647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:35.687179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:35.689071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:35.704925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386c79b-add4-451a-9fba-86fac7595c00 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:35.705117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 529/1045] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:35.706873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.711598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:35.742040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.742661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.742944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.743239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.743896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:35.744201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:35.745570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:35.758745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 517/1046] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:35.764857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.812219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:35.816893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff04e27-7e07-489a-9c75-623e0c0a6a3a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:35.853137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.853782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.854073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.854344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] PATH_INFO: `/v3/users/27ec9aca9c4a45f5aded9d84b4d01e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.868053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] RBAC: Authorizing `identity:get_user(user_id=27ec9aca9c4a45f5aded9d84b4d01e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:35.869512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:35.871225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 530/1047] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/users/27ec9aca9c4a45f5aded9d84b4d01e22 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:35.877110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.879629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526865d6-fef6-4e49-99f3-d965862008cc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:35.917978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:35.918735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:35.919022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:35.919328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:35.927432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:35.928885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:35.941549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:35.942477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:35.943445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 518/1048] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:35.948550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:35.981149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:35.988083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4587eb-f56b-482f-a728-02d44cde189d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:36.025023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab71a7-82fa-4751-b7a3-c799fb3c4270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:36.025628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab71a7-82fa-4751-b7a3-c799fb3c4270 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.025914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab71a7-82fa-4751-b7a3-c799fb3c4270 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.026195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab71a7-82fa-4751-b7a3-c799fb3c4270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.052663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:36.064609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecac782-7964-481b-8398-1f82ae2623e6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:36.098762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab71a7-82fa-4751-b7a3-c799fb3c4270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:36.100370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab71a7-82fa-4751-b7a3-c799fb3c4270 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:36.101558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:36.102177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.102458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.102731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.103470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:36.103807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:36.105200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:36.123227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 519/1049] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:36.124359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:36.208823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 531/1050] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:31:36.212216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e1fe-d8e9-40f7-be0d-98fef9f97779 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.212216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e1fe-d8e9-40f7-be0d-98fef9f97779 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.212216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e1fe-d8e9-40f7-be0d-98fef9f97779 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.212216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 532/1051] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:36.216166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e1fe-d8e9-40f7-be0d-98fef9f97779 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:36.223092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:36.233022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdf4cd-257f-40e4-839b-64caeb4a69a3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:36.266671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511f19c-cf2e-42ac-8699-6d21e01769cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:36.267302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f19c-cf2e-42ac-8699-6d21e01769cf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.267582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f19c-cf2e-42ac-8699-6d21e01769cf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.267842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f19c-cf2e-42ac-8699-6d21e01769cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.268567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511f19c-cf2e-42ac-8699-6d21e01769cf None admin] RBAC: Adding query filter params (role.id=b0feaa31560941bca79f97627f721c71, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=8e1ec869263246f58f5a32d855a3d302) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:36.268870 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511f19c-cf2e-42ac-8699-6d21e01769cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:36.270294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511f19c-cf2e-42ac-8699-6d21e01769cf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:36.274717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:36.291245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 520/1052] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:36 2022] GET /identity/v3/role_assignments?user.id=8e1ec869263246f58f5a32d855a3d302&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=b0feaa31560941bca79f97627f721c71 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:36.291245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896e043-49fb-4b51-9ba4-560a430791a8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.291245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896e043-49fb-4b51-9ba4-560a430791a8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.291245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896e043-49fb-4b51-9ba4-560a430791a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.297472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e896e043-49fb-4b51-9ba4-560a430791a8 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:36.298674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e896e043-49fb-4b51-9ba4-560a430791a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:36.334534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e1fe-d8e9-40f7-be0d-98fef9f97779 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:36.377387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:36.378041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.378319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.378588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.379738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] RBAC: Authorizing `identity:get_project(project_id=54dac7ce513b40f0a3202d2ec35956f3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:36.380405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 521/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:36.381196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:36.388157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 533/1054] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:31:36 2022] GET /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:36.389264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896e043-49fb-4b51-9ba4-560a430791a8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:36.475839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:36.493107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896e043-49fb-4b51-9ba4-560a430791a8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:36.527617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:36.528226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.528498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.528782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.529506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=54dac7ce513b40f0a3202d2ec35956f3, user.id=27ec9aca9c4a45f5aded9d84b4d01e22) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:36.529812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:36.531464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:36.537191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 522/1055] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:36 2022] GET /identity/v3/role_assignments?user.id=27ec9aca9c4a45f5aded9d84b4d01e22&scope.project.id=54dac7ce513b40f0a3202d2ec35956f3&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:36.663464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:36.779658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a739938-d3e7-4b09-b879-18ddcfe664bd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:36.805939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:36.827549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:36.828285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.828566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.828906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] PATH_INFO: `/v3/services/image` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.829507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:36.830909 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:36.840615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:36.841703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:31:36.842651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:36.847559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 534/1056] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Aug 24 06:31:36 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:36.848657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:36.958990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406017-a4f9-4ac9-8f35-f770f6cc499d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:36.994651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:36.995293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:36.995580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:36.995858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:36.996495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:36.996805 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:36.998187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:37.003610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 523/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:37.009613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:37.114346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92073f0-0919-4532-b0d3-1564903f7b65 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:37.150161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c5e1f-823a-4c66-bb4c-bec26f154711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:37.150788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c5e1f-823a-4c66-bb4c-bec26f154711 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:37.151071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c5e1f-823a-4c66-bb4c-bec26f154711 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:37.151382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c5e1f-823a-4c66-bb4c-bec26f154711 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:37.152027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c5e1f-823a-4c66-bb4c-bec26f154711 None admin] RBAC: Adding query filter params (type=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:37.152335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c5e1f-823a-4c66-bb4c-bec26f154711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:37.154032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c5e1f-823a-4c66-bb4c-bec26f154711 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:37.159392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 535/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:37.165276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:37.181121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:37.278210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd4024-f56a-4515-aa99-d6c4260330f6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:37.312997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73f345-28f7-48bb-afa2-07f5236dc985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:37.313608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f345-28f7-48bb-afa2-07f5236dc985 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:37.313893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f345-28f7-48bb-afa2-07f5236dc985 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:37.314169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73f345-28f7-48bb-afa2-07f5236dc985 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:37.314848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73f345-28f7-48bb-afa2-07f5236dc985 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:37.316738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73f345-28f7-48bb-afa2-07f5236dc985 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:37.340802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 524/1059] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:31:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:38.219371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0259b4-3a4d-4f5a-9b9b-6b6b8398b8bf None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:38.219371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0259b4-3a4d-4f5a-9b9b-6b6b8398b8bf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:38.219797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0259b4-3a4d-4f5a-9b9b-6b6b8398b8bf None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:38.221286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 536/1060] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:38.226484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb8ff0-64b3-4332-91d0-0465e78b9a8e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:38.226957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb8ff0-64b3-4332-91d0-0465e78b9a8e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:38.227045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb8ff0-64b3-4332-91d0-0465e78b9a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:38.246411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-88cb8ff0-64b3-4332-91d0-0465e78b9a8e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:38.247653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-88cb8ff0-64b3-4332-91d0-0465e78b9a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:38.312865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:38.327219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 525/1061] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:38.406957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9e79b-b4f5-467f-b9f2-c826134202e6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:38.407533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9e79b-b4f5-467f-b9f2-c826134202e6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:38.407893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9e79b-b4f5-467f-b9f2-c826134202e6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:38.411232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 537/1062] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:38.414897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01dbb-2b0a-4614-a8b5-ba78cf93f296 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:38.415205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01dbb-2b0a-4614-a8b5-ba78cf93f296 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:38.415503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01dbb-2b0a-4614-a8b5-ba78cf93f296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:38.435575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-85a01dbb-2b0a-4614-a8b5-ba78cf93f296 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:38.436941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-85a01dbb-2b0a-4614-a8b5-ba78cf93f296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:38.509815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 526/1063] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:38.594485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9e79b-b4f5-467f-b9f2-c826134202e6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:38.660725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:38.713946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9e79b-b4f5-467f-b9f2-c826134202e6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:38.748810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:38.749478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:38.749853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:38.750037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:38.761077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:38.765948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:38.766918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:38.772215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 538/1064] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 06:31:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:38.773584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a01dbb-2b0a-4614-a8b5-ba78cf93f296 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:38.774164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:38.884257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a01dbb-2b0a-4614-a8b5-ba78cf93f296 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:38.884628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2737c76-9e93-4eb7-8b24-76b984790eeb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:38.898402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:38.911796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:38.922033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:38.922766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:38.923151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:38.923535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:38.924022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:38.924640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:38.924922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:38.925197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:38.925872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:38.926176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:38.927572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:38.934762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:38.941504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 527/1065] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 06:31:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:38.941663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:38.943041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:38.944078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 539/1066] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:38.946948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:38.947078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.057943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc969217-8d44-4276-a423-45ca60fc3094 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.059362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8463d-15fd-43b2-92e7-526f1756f777 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.093646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.094182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.094304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.094538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.094813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.094918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.095110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.095387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.095553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:39.095824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.097248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.106045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 540/1067] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:31:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:39.107577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.109125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.112173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:39.126801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:39.128232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:39.129343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:39.129343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:39.129343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:39.132900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 528/1068] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:39.134048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.150963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.184138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.236907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c5f68-00c2-4555-92d1-5163399a8512 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.252129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e3b44-127a-402a-9c7e-f523307b4837 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.274129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4a977-8cc7-4086-9cd4-54ca246f5630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.274771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4a977-8cc7-4086-9cd4-54ca246f5630 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.275075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4a977-8cc7-4086-9cd4-54ca246f5630 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.275520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4a977-8cc7-4086-9cd4-54ca246f5630 None admin] PATH_INFO: `/v3/users/27ec9aca9c4a45f5aded9d84b4d01e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.276765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4a977-8cc7-4086-9cd4-54ca246f5630 None admin] RBAC: Authorizing `identity:get_user(user_id=27ec9aca9c4a45f5aded9d84b4d01e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.278238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4a977-8cc7-4086-9cd4-54ca246f5630 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.279916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 541/1069] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/users/27ec9aca9c4a45f5aded9d84b4d01e22 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:39.281206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0d9e8-515a-4c03-b53e-f9c112eb9a2f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.281489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0d9e8-515a-4c03-b53e-f9c112eb9a2f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.281765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0d9e8-515a-4c03-b53e-f9c112eb9a2f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.283125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 542/1070] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:39.285759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0d9e8-515a-4c03-b53e-f9c112eb9a2f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.293000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563b014-1e63-4e6c-bfc4-fad83f4992c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.293617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563b014-1e63-4e6c-bfc4-fad83f4992c5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.293896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563b014-1e63-4e6c-bfc4-fad83f4992c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.294167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563b014-1e63-4e6c-bfc4-fad83f4992c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.294813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563b014-1e63-4e6c-bfc4-fad83f4992c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:39.295111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563b014-1e63-4e6c-bfc4-fad83f4992c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.296588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563b014-1e63-4e6c-bfc4-fad83f4992c5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.308791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 529/1071] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:39.311530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06da9e-33ae-4ee0-b2e0-0faa0b9f75f5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.311530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06da9e-33ae-4ee0-b2e0-0faa0b9f75f5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.311937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06da9e-33ae-4ee0-b2e0-0faa0b9f75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.330849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ce06da9e-33ae-4ee0-b2e0-0faa0b9f75f5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:39.332104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ce06da9e-33ae-4ee0-b2e0-0faa0b9f75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:39.390670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0d9e8-515a-4c03-b53e-f9c112eb9a2f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.392292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.408128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 530/1072] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:39.408979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06da9e-33ae-4ee0-b2e0-0faa0b9f75f5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.432261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.432889 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.433162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.433457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.434588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] RBAC: Authorizing `identity:get_project(project_id=54dac7ce513b40f0a3202d2ec35956f3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.436204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.438537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 543/1073] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:39.444640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.446652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.525348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06da9e-33ae-4ee0-b2e0-0faa0b9f75f5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.549286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486d296-f6a7-4ad6-bfe8-d7867fde2335 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.560899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.561597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.562293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.562597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.571568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.573029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:39.585342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:39.586465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:39.587479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 531/1074] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:39.592022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.592724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.592900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.592995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.593270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3/users/27ec9aca9c4a45f5aded9d84b4d01e22/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.595215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=54dac7ce513b40f0a3202d2ec35956f3, user_id=27ec9aca9c4a45f5aded9d84b4d01e22, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.596687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.615267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 544/1075] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:31:39 2022] PUT /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3/users/27ec9aca9c4a45f5aded9d84b4d01e22/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:31:39.688736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.707880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.723951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2415ab-97aa-4316-8254-b2e6ef5a94bb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.724946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.769003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.769631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.769918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.770210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.771060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:39.771429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.772836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.799885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 532/1076] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:39.805316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:39.807173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff815d-72b3-4404-b5b9-eb4aacad1ec3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.846878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.847449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.847730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.848003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.916240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064de7c-f092-4042-b298-70357a60cb7a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:39.923436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.924912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.957264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.958259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:39.958907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:39.959217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:39.959642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:39.971013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:39.973008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:39.980811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:39.989453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:39.990457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:39.991525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:39.991525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:39.991525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:39.994165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 533/1077] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:39.999437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:40.043244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 545/1078] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:40.045186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:40.118571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d594999-6aaa-46eb-a673-efe15077c5db None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:40.149503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce0c57-0b6a-450a-a47d-52bad35c3bd0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:40.155453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:40.156071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:40.156349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:40.156620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:40.157381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:40.157683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:40.159290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:40.177111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 534/1079] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:40.178156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:40.180501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:40.189076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:40.189831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:40.190109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:40.190788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:40.201169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:40.202856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:40.217667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:40.218638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:40.219612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:40.219612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:40.219612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:40.219612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 546/1080] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:40.225136 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:40.248622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:40.300707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871cd56-176b-4fd1-b870-bf4e219536e4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:40.337310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:40.337964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:40.338244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:40.338522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:40.341826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9266ca-1455-4e18-8942-06d743a7c049 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:40.345809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:40.347361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:40.358754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:40.359734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:40.360684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 535/1081] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:40.365709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:40.380337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:40.381037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:40.381329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:40.381609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:40.382266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:40.382574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:40.384088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:40.407234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 547/1082] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:40.488116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af460ef-e377-4409-82e6-825e14e0f02d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:40.507244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:40.526474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d0144-41c7-472c-9c53-ec0dbccaf5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:40.527095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d0144-41c7-472c-9c53-ec0dbccaf5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:40.527477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d0144-41c7-472c-9c53-ec0dbccaf5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:40.527752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d0144-41c7-472c-9c53-ec0dbccaf5e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:40.528460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d0144-41c7-472c-9c53-ec0dbccaf5e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:40.528763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d0144-41c7-472c-9c53-ec0dbccaf5e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:40.530361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d0144-41c7-472c-9c53-ec0dbccaf5e4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:40.544343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 536/1083] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:40.550971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:40.599973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:40.661934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68612274-f1ee-462d-be99-9a81cda6c276 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:40.700204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aeada-4656-4b2c-9c00-d5e59cba0da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:40.700910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aeada-4656-4b2c-9c00-d5e59cba0da7 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:40.701184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aeada-4656-4b2c-9c00-d5e59cba0da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:40.701455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aeada-4656-4b2c-9c00-d5e59cba0da7 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/8e1ec869263246f58f5a32d855a3d302/roles/b0feaa31560941bca79f97627f721c71` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:40.703862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aeada-4656-4b2c-9c00-d5e59cba0da7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=8e1ec869263246f58f5a32d855a3d302, role_id=b0feaa31560941bca79f97627f721c71)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:40.705392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aeada-4656-4b2c-9c00-d5e59cba0da7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:40.720359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 548/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:31:40 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/8e1ec869263246f58f5a32d855a3d302/roles/b0feaa31560941bca79f97627f721c71 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:31:41.535441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c414496-c0fa-45f3-86a6-68e05abf39e7 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:41.536008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c414496-c0fa-45f3-86a6-68e05abf39e7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:41.536008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c414496-c0fa-45f3-86a6-68e05abf39e7 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:41.537367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 537/1085] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:41.542769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f7e8d-da31-4537-ad1c-7f25d20a03eb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:41.543243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f7e8d-da31-4537-ad1c-7f25d20a03eb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:41.543359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f7e8d-da31-4537-ad1c-7f25d20a03eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:41.562514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-163f7e8d-da31-4537-ad1c-7f25d20a03eb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:41.563745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-163f7e8d-da31-4537-ad1c-7f25d20a03eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:41.636374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 549/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:41.900138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c414496-c0fa-45f3-86a6-68e05abf39e7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:42.009061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c414496-c0fa-45f3-86a6-68e05abf39e7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:42.044186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:42.044822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.045105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.045379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.055641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:42.060326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:42.061274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:42.062841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 538/1087] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:31:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:42.068444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f7e8d-da31-4537-ad1c-7f25d20a03eb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:42.100216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:42.171096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f7e8d-da31-4537-ad1c-7f25d20a03eb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:42.206451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:42.207061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.207372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.207647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.208331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:42.208625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:42.209995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:42.219664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 550/1088] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:42.226073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:42.268532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:42.329933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e728f-e59a-48ae-8bb8-4e7db4ecbbfb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:42.363309 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42710472-3702-4276-8fbd-f20d7b84bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:42.363946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710472-3702-4276-8fbd-f20d7b84bb99 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.364228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710472-3702-4276-8fbd-f20d7b84bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.364505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42710472-3702-4276-8fbd-f20d7b84bb99 None admin] PATH_INFO: `/v3/users/27ec9aca9c4a45f5aded9d84b4d01e22` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.365653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42710472-3702-4276-8fbd-f20d7b84bb99 None admin] RBAC: Authorizing `identity:get_user(user_id=27ec9aca9c4a45f5aded9d84b4d01e22)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:42.367108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42710472-3702-4276-8fbd-f20d7b84bb99 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:42.368748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 539/1089] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:42 2022] GET /identity/v3/users/27ec9aca9c4a45f5aded9d84b4d01e22 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:42.375148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:42.471042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:42.478461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e69d5b-5628-4f34-b8fa-158480579426 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:42.515658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:42.516285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.516568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.516838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.587978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:42.589474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:42.603080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efee351-5988-43f3-aeac-31e0c9213c5c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.603636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efee351-5988-43f3-aeac-31e0c9213c5c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.603762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efee351-5988-43f3-aeac-31e0c9213c5c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.605128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 540/1090] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:31:42.610417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe0f3-d4d5-4b96-afab-836ad7d1373a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.610711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe0f3-d4d5-4b96-afab-836ad7d1373a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.610991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe0f3-d4d5-4b96-afab-836ad7d1373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.629998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-75dbe0f3-d4d5-4b96-afab-836ad7d1373a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:42.631230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-75dbe0f3-d4d5-4b96-afab-836ad7d1373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:42.684049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:42.692841 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 551/1091] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:42.699731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:42.707532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 541/1092] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:42.800264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00e81-204b-4545-b029-4cd7563f09f9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:42.835069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:42.835732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.836019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.836296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.837441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] RBAC: Authorizing `identity:get_project(project_id=54dac7ce513b40f0a3202d2ec35956f3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:42.838888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:42.840574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 552/1093] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:31:42 2022] GET /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:42.846926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe0f3-d4d5-4b96-afab-836ad7d1373a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:42.881773 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:42.949955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe0f3-d4d5-4b96-afab-836ad7d1373a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:42.967332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:42.988070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:42.988691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:42.988971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:42.989242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:42.989975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=54dac7ce513b40f0a3202d2ec35956f3, user.id=27ec9aca9c4a45f5aded9d84b4d01e22) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:42.990274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:42.991748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:42.997653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 542/1094] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:42 2022] GET /identity/v3/role_assignments?user.id=27ec9aca9c4a45f5aded9d84b4d01e22&scope.project.id=54dac7ce513b40f0a3202d2ec35956f3&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:43.035144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:43.077929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1510106-3ff4-40cf-b2f7-d44c06258326 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:43.116442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:43.117098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:43.117382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:43.117657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:43.129343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:31:43.134091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:43.135266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:43.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 553/1095] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Aug 24 06:31:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:31:43.141852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:43.245296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:43.248933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93021-6209-4d37-b02d-7fdd4b15c594 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:43.283634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:43.284247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:43.284533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:43.284801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:43.285473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:43.285778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:43.287147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:43.295338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 543/1096] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 06:31:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:43.301498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:43.403591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cb0fd-5ddb-4a47-b3d3-292019e0fe98 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:43.428721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:43.442445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:43.443088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:43.443399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:43.443672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:43.453926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:43.455403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:43.469348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:43.475489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:43.476495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:43.476495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:43.476495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:43.476495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 554/1097] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:43.476728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:43.575808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147f35-8a68-4001-b2f7-9c6b1ecab952 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:43.611281 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:43.611922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:43.612153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:43.612414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:43.613050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:43.613350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:43.614707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:43.626747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 544/1098] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:43.632519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:43.736032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7440c-66b0-4229-ac4f-e3f3b33dcccf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:43.773160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:43.773802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:43.774086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:43.774362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:43.782626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:43.784126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:31:43.795437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:43.796424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:31:43.797384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:31:43.799380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 555/1099] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Aug 24 06:31:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:43.802890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:43.828427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:43.905526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec99a64-ea26-4ded-af4c-db25294ee930 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:43.940000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:43.940973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:43.941256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:43.941514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:43.942248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:43.942553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:43.944033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:43.961165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 545/1100] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Aug 24 06:31:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:43.967317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:44.005049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:44.077805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326793ed-0ccf-418b-87be-0a37df8df69c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:44.116797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:44.117417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:44.117702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:44.117978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:44.191173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:44.192671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:44.227639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:44.295080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 556/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:44.301094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:44.381912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:44.404135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303991a1-4518-41e0-81c8-aff0fc0ea5eb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:44.439719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:44.440711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:44.440987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:44.441256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:44.451441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:44.453432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:31:44.468051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:44.469064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:31:44.469998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:31:44.469998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:31:44.469998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:44.469998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 546/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:44.475471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:44.579341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f010e-29c8-4cfd-a13c-461cf04a1abf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:44.614467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:44.615073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:44.615381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:44.615662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:44.616307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:44.616612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:44.617976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:44.619460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:44.631491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 557/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:44.637332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:44.740816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2f930-5ef6-4dae-8d5f-d112ad701b7c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:44.770472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:44.776371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c696f8-bfc6-4fc2-8794-a20c72fb58b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:44.777007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c696f8-bfc6-4fc2-8794-a20c72fb58b5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:44.777288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c696f8-bfc6-4fc2-8794-a20c72fb58b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:44.777565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c696f8-bfc6-4fc2-8794-a20c72fb58b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:44.784219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c696f8-bfc6-4fc2-8794-a20c72fb58b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:44.785667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c696f8-bfc6-4fc2-8794-a20c72fb58b5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-84c696f8-bfc6-4fc2-8794-a20c72fb58b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:31:44.796747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:31:44.797698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:31:44.798679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 547/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:31:44.803888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:44.907888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8803b5-a7dd-411f-a990-1fdf73f5339e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:44.943961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:44.944568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:44.944853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:44.945125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:44.945825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:44.946129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:44.947683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:44.948686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ca4a-c0be-48d5-8e97-b78b3ae14f0d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:44.948979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ca4a-c0be-48d5-8e97-b78b3ae14f0d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:44.949295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ca4a-c0be-48d5-8e97-b78b3ae14f0d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:44.950902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 548/1105] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:44.956250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165af129-0d09-48f5-9ac4-e93059eb2227 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:44.956540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165af129-0d09-48f5-9ac4-e93059eb2227 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:44.956817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165af129-0d09-48f5-9ac4-e93059eb2227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:44.961699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 558/1106] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:44.967517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:44.975609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-165af129-0d09-48f5-9ac4-e93059eb2227 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:44.976822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-165af129-0d09-48f5-9ac4-e93059eb2227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:45.024812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:45.046077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 549/1107] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:45.073191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fbca4-b37e-4a8c-958a-fe70b3b7e033 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:45.107814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:45.108583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:45.109233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:45.109514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:45.110258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] RBAC: Adding query filter params (role.id=b0feaa31560941bca79f97627f721c71, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=8e1ec869263246f58f5a32d855a3d302) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:45.110553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:45.112027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:45.117832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 559/1108] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:44 2022] GET /identity/v3/role_assignments?user.id=8e1ec869263246f58f5a32d855a3d302&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=b0feaa31560941bca79f97627f721c71 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:31:45.313557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165af129-0d09-48f5-9ac4-e93059eb2227 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:45.415388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165af129-0d09-48f5-9ac4-e93059eb2227 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:45.440645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:45.454606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cf2dc-a36b-4508-8830-12c75898f5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:45.455288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cf2dc-a36b-4508-8830-12c75898f5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:45.455574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cf2dc-a36b-4508-8830-12c75898f5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:45.455850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cf2dc-a36b-4508-8830-12c75898f5f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:45.457041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cf2dc-a36b-4508-8830-12c75898f5f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:45.458498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cf2dc-a36b-4508-8830-12c75898f5f2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:45.460164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 550/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:45.466096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:45.581587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b3790-5220-4540-9322-92bb58fb418a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:45.599048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:45.616571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ddae-d8df-4d78-8dc1-2ea4487f439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:45.617193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ddae-d8df-4d78-8dc1-2ea4487f439e None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:45.617475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ddae-d8df-4d78-8dc1-2ea4487f439e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:45.617749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ddae-d8df-4d78-8dc1-2ea4487f439e None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:45.618401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ddae-d8df-4d78-8dc1-2ea4487f439e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:45.619909 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ddae-d8df-4d78-8dc1-2ea4487f439e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:45.642758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 560/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:31:45 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:47.579149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035fc5-1e8e-4f8d-b9c1-1390f48e3151 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:47.580159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035fc5-1e8e-4f8d-b9c1-1390f48e3151 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:47.580159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035fc5-1e8e-4f8d-b9c1-1390f48e3151 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:47.582321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 551/1111] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:47.591177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca32225-94d5-4d26-a32a-cfe07241cc4e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:47.591664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca32225-94d5-4d26-a32a-cfe07241cc4e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:47.591793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca32225-94d5-4d26-a32a-cfe07241cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:47.612467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-cca32225-94d5-4d26-a32a-cfe07241cc4e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:47.613679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cca32225-94d5-4d26-a32a-cfe07241cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:47.694127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 561/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:47.965880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035fc5-1e8e-4f8d-b9c1-1390f48e3151 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:48.087910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035fc5-1e8e-4f8d-b9c1-1390f48e3151 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:48.123717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:48.124855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:48.125134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:48.125438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] PATH_INFO: `/v3/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:48.126940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] RBAC: Authorizing `identity:get_role(role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:48.128315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:48.129668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 552/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:47 2022] GET /identity/v3/roles/c37a4d448aa147ebb487e7de2099e1db => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:48.135722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca32225-94d5-4d26-a32a-cfe07241cc4e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:48.236601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca32225-94d5-4d26-a32a-cfe07241cc4e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:48.238932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:48.270943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:48.271575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:48.271886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:48.272152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] PATH_INFO: `/v3/users/39d8eeb275fe4986823b382ca4030395` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:48.285742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] RBAC: Authorizing `identity:get_user(user_id=39d8eeb275fe4986823b382ca4030395)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:48.287213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:48.288911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 562/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:48 2022] GET /identity/v3/users/39d8eeb275fe4986823b382ca4030395 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:48.295156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:48.361675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:48.411410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f045cc-9b44-4648-bc78-5480335d96cd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:48.445931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:48.446522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:48.446798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:48.447069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:48.517883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:48.519351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:48.582160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:48.627758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 553/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:48.633955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:48.741910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a28d3-5165-444f-b070-c5db6b7f33a4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:48.779890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744df6b-ed9d-4d0d-9f71-286220f924be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:48.780495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744df6b-ed9d-4d0d-9f71-286220f924be None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:48.780773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744df6b-ed9d-4d0d-9f71-286220f924be None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:48.781074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744df6b-ed9d-4d0d-9f71-286220f924be None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:48.782291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744df6b-ed9d-4d0d-9f71-286220f924be None admin] RBAC: Authorizing `identity:get_project(project_id=54dac7ce513b40f0a3202d2ec35956f3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:48.783756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744df6b-ed9d-4d0d-9f71-286220f924be None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:48.785399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 563/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:31:48 2022] GET /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:48.792662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:48.905829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2f32-1ee8-4651-ac0f-ee1e5128d468 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:48.941973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8c1c5-47f6-45ee-9c4e-b2fc5bb9f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:48.942574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8c1c5-47f6-45ee-9c4e-b2fc5bb9f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:48.942853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8c1c5-47f6-45ee-9c4e-b2fc5bb9f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:48.943128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8c1c5-47f6-45ee-9c4e-b2fc5bb9f8e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:48.943900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8c1c5-47f6-45ee-9c4e-b2fc5bb9f8e9 None admin] RBAC: Adding query filter params (role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=54dac7ce513b40f0a3202d2ec35956f3, user.id=39d8eeb275fe4986823b382ca4030395) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:48.944204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8c1c5-47f6-45ee-9c4e-b2fc5bb9f8e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:48.945650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8c1c5-47f6-45ee-9c4e-b2fc5bb9f8e9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:48.952918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 554/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:48 2022] GET /identity/v3/role_assignments?user.id=39d8eeb275fe4986823b382ca4030395&scope.project.id=54dac7ce513b40f0a3202d2ec35956f3&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:50.900513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c798c5-5d35-4e5f-a9f4-0978e87825a4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:50.901127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c798c5-5d35-4e5f-a9f4-0978e87825a4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:50.901127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c798c5-5d35-4e5f-a9f4-0978e87825a4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:50.902423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 564/1118] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:50.908941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b1a7d-9f6a-4f34-9460-e9fd57aa7c63 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:50.909647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b1a7d-9f6a-4f34-9460-e9fd57aa7c63 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:50.909759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b1a7d-9f6a-4f34-9460-e9fd57aa7c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:50.933199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-190b1a7d-9f6a-4f34-9460-e9fd57aa7c63 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:50.934388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-190b1a7d-9f6a-4f34-9460-e9fd57aa7c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:50.951084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:51.006671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 555/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:51.279532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c798c5-5d35-4e5f-a9f4-0978e87825a4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:51.315150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:51.390465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c798c5-5d35-4e5f-a9f4-0978e87825a4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:51.425215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:51.425837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:51.426125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:51.426415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] PATH_INFO: `/v3/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:51.427940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] RBAC: Authorizing `identity:get_role(role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:51.429328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:51.430679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 565/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:51 2022] GET /identity/v3/roles/c37a4d448aa147ebb487e7de2099e1db => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:51.437160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b1a7d-9f6a-4f34-9460-e9fd57aa7c63 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:51.546515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b1a7d-9f6a-4f34-9460-e9fd57aa7c63 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:51.582268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:51.582889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:51.583175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:51.583482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] PATH_INFO: `/v3/users/39d8eeb275fe4986823b382ca4030395` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:51.584662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] RBAC: Authorizing `identity:get_user(user_id=39d8eeb275fe4986823b382ca4030395)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:51.586366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:51.588293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 556/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:51 2022] GET /identity/v3/users/39d8eeb275fe4986823b382ca4030395 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:51.598044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:51.709037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:51.713009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b2e1-1a5d-4a17-ac9c-46ecf1e55100 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:51.748425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ba65a-11a8-42ff-a00d-e54c1089d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:51.749531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba65a-11a8-42ff-a00d-e54c1089d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:51.749816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba65a-11a8-42ff-a00d-e54c1089d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:51.750088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba65a-11a8-42ff-a00d-e54c1089d7e9 None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:51.751297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ba65a-11a8-42ff-a00d-e54c1089d7e9 None admin] RBAC: Authorizing `identity:get_project(project_id=54dac7ce513b40f0a3202d2ec35956f3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:51.752747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ba65a-11a8-42ff-a00d-e54c1089d7e9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:51.754373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 566/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:31:51 2022] GET /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:51.764078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:51.787316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:51.876427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad4630-2d61-440b-85f5-3c33a803b954 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:51.910887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb7086-61c4-4cb2-8f7c-3254e4de8b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:51.911541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb7086-61c4-4cb2-8f7c-3254e4de8b6b None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:51.911843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb7086-61c4-4cb2-8f7c-3254e4de8b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:51.912114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb7086-61c4-4cb2-8f7c-3254e4de8b6b None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3/users/39d8eeb275fe4986823b382ca4030395/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:51.914521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb7086-61c4-4cb2-8f7c-3254e4de8b6b None admin] RBAC: Authorizing `identity:create_grant(project_id=54dac7ce513b40f0a3202d2ec35956f3, user_id=39d8eeb275fe4986823b382ca4030395, role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:51.915981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb7086-61c4-4cb2-8f7c-3254e4de8b6b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:51.929547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 557/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:31:51 2022] PUT /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3/users/39d8eeb275fe4986823b382ca4030395/roles/c37a4d448aa147ebb487e7de2099e1db => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:31:53.815373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77de4b0-8f72-4dcd-b173-46905db0cb90 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:53.816718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77de4b0-8f72-4dcd-b173-46905db0cb90 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:53.816718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77de4b0-8f72-4dcd-b173-46905db0cb90 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:53.818822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 567/1124] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:53.828201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf23f-ca5e-4c16-9845-2faa0622d3b4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:53.828690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf23f-ca5e-4c16-9845-2faa0622d3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:53.828781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcf23f-ca5e-4c16-9845-2faa0622d3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:53.848535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a2fcf23f-ca5e-4c16-9845-2faa0622d3b4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:53.849718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a2fcf23f-ca5e-4c16-9845-2faa0622d3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:53.922655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 558/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:54.188517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77de4b0-8f72-4dcd-b173-46905db0cb90 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:54.314856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77de4b0-8f72-4dcd-b173-46905db0cb90 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:54.349380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:54.350000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:54.350282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:54.350557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] PATH_INFO: `/v3/roles/c37a4d448aa147ebb487e7de2099e1db` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:54.351976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] RBAC: Authorizing `identity:get_role(role_id=c37a4d448aa147ebb487e7de2099e1db)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:54.353357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:54.354681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 568/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:54 2022] GET /identity/v3/roles/c37a4d448aa147ebb487e7de2099e1db => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:54.363397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf23f-ca5e-4c16-9845-2faa0622d3b4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:54.376607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:54.492862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcf23f-ca5e-4c16-9845-2faa0622d3b4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:54.542228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:54.543276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:54.543771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:54.544230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] PATH_INFO: `/v3/users/39d8eeb275fe4986823b382ca4030395` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:54.546223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] RBAC: Authorizing `identity:get_user(user_id=39d8eeb275fe4986823b382ca4030395)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:54.547729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:54.549338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 559/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:31:54 2022] GET /identity/v3/users/39d8eeb275fe4986823b382ca4030395 => generated 286 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:54.555402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:54.593016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:54.669704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624cc5-0970-4b95-a487-5337c5390749 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:54.712915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:54.713527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:54.718481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:54.719613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:54.801382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:54.802762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:54.839185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:54.917978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 569/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:31:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:31:54.924288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:55.015628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:55.043000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1cc9-2258-4f39-94bb-ed4af7feb02b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:55.079287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4757d69-ea4a-4a60-ade2-0638aa3ab9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:55.079867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4757d69-ea4a-4a60-ade2-0638aa3ab9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:55.080146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4757d69-ea4a-4a60-ade2-0638aa3ab9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:55.080414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4757d69-ea4a-4a60-ade2-0638aa3ab9b2 None admin] PATH_INFO: `/v3/projects/54dac7ce513b40f0a3202d2ec35956f3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:55.081525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4757d69-ea4a-4a60-ade2-0638aa3ab9b2 None admin] RBAC: Authorizing `identity:get_project(project_id=54dac7ce513b40f0a3202d2ec35956f3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:55.082960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4757d69-ea4a-4a60-ade2-0638aa3ab9b2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:55.084638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 560/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:31:54 2022] GET /identity/v3/projects/54dac7ce513b40f0a3202d2ec35956f3 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:55.091431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:55.202239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832033f3-4049-43d3-9b38-0447d52e58c9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:55.241016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbf4a0-9b14-460c-8cc3-0776afd63f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:55.241631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbf4a0-9b14-460c-8cc3-0776afd63f94 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:55.241916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbf4a0-9b14-460c-8cc3-0776afd63f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:55.242188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbf4a0-9b14-460c-8cc3-0776afd63f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:55.242926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbf4a0-9b14-460c-8cc3-0776afd63f94 None admin] RBAC: Adding query filter params (role.id=c37a4d448aa147ebb487e7de2099e1db, scope.project.id=54dac7ce513b40f0a3202d2ec35956f3, user.id=39d8eeb275fe4986823b382ca4030395) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:31:55.243277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbf4a0-9b14-460c-8cc3-0776afd63f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:55.244730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbf4a0-9b14-460c-8cc3-0776afd63f94 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:55.246511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:55.251573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 570/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:31:55 2022] GET /identity/v3/role_assignments?user.id=39d8eeb275fe4986823b382ca4030395&scope.project.id=54dac7ce513b40f0a3202d2ec35956f3&role.id=c37a4d448aa147ebb487e7de2099e1db => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:57.165812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8652b-22a6-4fd5-a9f6-8009f5dba66d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:57.166430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8652b-22a6-4fd5-a9f6-8009f5dba66d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:57.166430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8652b-22a6-4fd5-a9f6-8009f5dba66d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:57.167778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 561/1131] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:57.174449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dcaa3-d12d-4e44-ad9b-78feb0d81859 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:57.175257 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dcaa3-d12d-4e44-ad9b-78feb0d81859 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:57.175429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dcaa3-d12d-4e44-ad9b-78feb0d81859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:57.198897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f04dcaa3-d12d-4e44-ad9b-78feb0d81859 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:57.200162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f04dcaa3-d12d-4e44-ad9b-78feb0d81859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:57.270760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 571/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:31:57.538786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8652b-22a6-4fd5-a9f6-8009f5dba66d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:57.666735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8652b-22a6-4fd5-a9f6-8009f5dba66d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:57.703976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73dcff-6d76-41ec-8524-242e82a158da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:57.704597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73dcff-6d76-41ec-8524-242e82a158da None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:57.704875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73dcff-6d76-41ec-8524-242e82a158da None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:57.705152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73dcff-6d76-41ec-8524-242e82a158da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:57.706391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73dcff-6d76-41ec-8524-242e82a158da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:57.707961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73dcff-6d76-41ec-8524-242e82a158da None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:57.710784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 562/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:31:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:31:57.716939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dcaa3-d12d-4e44-ad9b-78feb0d81859 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:31:57.818376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dcaa3-d12d-4e44-ad9b-78feb0d81859 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:31:57.847682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:31:57.858394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62b5b3-b15c-4f2b-9f86-dedc62a3ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:31:57.859119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62b5b3-b15c-4f2b-9f86-dedc62a3ca4f None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:57.859315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62b5b3-b15c-4f2b-9f86-dedc62a3ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:57.859602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62b5b3-b15c-4f2b-9f86-dedc62a3ca4f None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:57.860255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62b5b3-b15c-4f2b-9f86-dedc62a3ca4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:31:57.861710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62b5b3-b15c-4f2b-9f86-dedc62a3ca4f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:31:57.899162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 572/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:31:57 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:31:59.814721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c4b85-fb21-4631-883a-f9ff0c89525c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:59.815998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c4b85-fb21-4631-883a-f9ff0c89525c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:59.815998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c4b85-fb21-4631-883a-f9ff0c89525c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:59.818133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 563/1135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:31:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:31:59.823720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53cdd2-7092-4962-9fd5-9ad6bd8d52d4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:31:59.824157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53cdd2-7092-4962-9fd5-9ad6bd8d52d4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:31:59.824266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53cdd2-7092-4962-9fd5-9ad6bd8d52d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:31:59.843187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ec53cdd2-7092-4962-9fd5-9ad6bd8d52d4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:31:59.844413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ec53cdd2-7092-4962-9fd5-9ad6bd8d52d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:31:59.914666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 573/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:31:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:00.181765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c4b85-fb21-4631-883a-f9ff0c89525c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:00.206302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:00.299954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c4b85-fb21-4631-883a-f9ff0c89525c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:00.335529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9b5f-3734-4571-acf3-2fe0c2153d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:00.336154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9b5f-3734-4571-acf3-2fe0c2153d0d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:00.336437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9b5f-3734-4571-acf3-2fe0c2153d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:00.336717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9b5f-3734-4571-acf3-2fe0c2153d0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:00.338049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9b5f-3734-4571-acf3-2fe0c2153d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:00.339530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9b5f-3734-4571-acf3-2fe0c2153d0d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:00.341102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 564/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:32:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:00.350361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53cdd2-7092-4962-9fd5-9ad6bd8d52d4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:00.455967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53cdd2-7092-4962-9fd5-9ad6bd8d52d4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:00.467776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:00.499210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7afbb6-4e48-464e-97d6-7e62e668bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:00.499844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7afbb6-4e48-464e-97d6-7e62e668bb6b None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:00.500123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7afbb6-4e48-464e-97d6-7e62e668bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:00.500393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7afbb6-4e48-464e-97d6-7e62e668bb6b None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:00.501050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7afbb6-4e48-464e-97d6-7e62e668bb6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:00.502498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7afbb6-4e48-464e-97d6-7e62e668bb6b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:00.527425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 574/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:32:00 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:32:02.498103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c41fd-23b4-46c2-bd9e-79a4834aabbf None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:02.498948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c41fd-23b4-46c2-bd9e-79a4834aabbf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:02.498948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c41fd-23b4-46c2-bd9e-79a4834aabbf None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:02.500824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 565/1139] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:02.509793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301f67-a89a-4c98-a78c-6bedd4202bc3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:02.510236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301f67-a89a-4c98-a78c-6bedd4202bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:02.510368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301f67-a89a-4c98-a78c-6bedd4202bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:02.540198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-95301f67-a89a-4c98-a78c-6bedd4202bc3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:02.542028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-95301f67-a89a-4c98-a78c-6bedd4202bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:02.625375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 575/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:02.898762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c41fd-23b4-46c2-bd9e-79a4834aabbf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:02.989283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:03.016771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c41fd-23b4-46c2-bd9e-79a4834aabbf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:03.051215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:03.051865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:03.052150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:03.052427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:03.062971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:32:03.068090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:32:03.069274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:32:03.070233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 566/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:32:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:32:03.077167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301f67-a89a-4c98-a78c-6bedd4202bc3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:03.185061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:03.192596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301f67-a89a-4c98-a78c-6bedd4202bc3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:03.228766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:03.229379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:03.229665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:03.229944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:03.230614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:03.230920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:03.232347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:03.240472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 576/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:32:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:32:03.246573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:03.354482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e6494-58f7-416f-9de1-e3bf3ae37c35 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:03.389525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:03.390378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:03.390530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:03.390776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] PATH_INFO: `/v3/users/b03c4814e4bb4a3099cf8b244a20a97f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:03.402457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:03.405994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] RBAC: Authorizing `identity:get_user(user_id=b03c4814e4bb4a3099cf8b244a20a97f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:03.408811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:03.411564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 567/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:32:03 2022] GET /identity/v3/users/b03c4814e4bb4a3099cf8b244a20a97f => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:03.422203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:03.543373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac064c-2fa1-456c-b7b5-4893a31322da None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:03.587471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:03.588091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:03.588373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:03.588650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:03.619238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:03.660542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:03.662009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:03.767544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 577/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:32:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:32:03.773831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:03.883646 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b290d-1f09-424c-9ef4-5ae3c524cbbb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:03.921992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e41f9-fabe-47e0-b625-3c24505862f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:03.922620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e41f9-fabe-47e0-b625-3c24505862f3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:03.922911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e41f9-fabe-47e0-b625-3c24505862f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:03.923235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e41f9-fabe-47e0-b625-3c24505862f3 None admin] PATH_INFO: `/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:03.924465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e41f9-fabe-47e0-b625-3c24505862f3 None admin] RBAC: Authorizing `identity:get_project(project_id=adaa4d973454487ea7dbc338bc8e8bfa)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:03.925910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e41f9-fabe-47e0-b625-3c24505862f3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:03.927604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 568/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:32:03 2022] GET /identity/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:03.934128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:03.997103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:04.039999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17192075-6a65-4d6f-99c5-603ea7df5430 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:04.075379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada94e0-d6f1-4f9c-ac8b-812e2839f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:04.075956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada94e0-d6f1-4f9c-ac8b-812e2839f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:04.076242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada94e0-d6f1-4f9c-ac8b-812e2839f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:04.076516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada94e0-d6f1-4f9c-ac8b-812e2839f5f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:04.077268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada94e0-d6f1-4f9c-ac8b-812e2839f5f0 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=adaa4d973454487ea7dbc338bc8e8bfa, user.id=b03c4814e4bb4a3099cf8b244a20a97f) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:04.078052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada94e0-d6f1-4f9c-ac8b-812e2839f5f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:04.079544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada94e0-d6f1-4f9c-ac8b-812e2839f5f0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:04.084871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 578/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:32:03 2022] GET /identity/v3/role_assignments?user.id=b03c4814e4bb4a3099cf8b244a20a97f&scope.project.id=adaa4d973454487ea7dbc338bc8e8bfa&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:05.999022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42928fd7-698a-49bc-b574-a2c3a46b0b9e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:05.999640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42928fd7-698a-49bc-b574-a2c3a46b0b9e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:06.000191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42928fd7-698a-49bc-b574-a2c3a46b0b9e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:06.002567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 569/1147] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:06.009685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69f59a-c166-4435-addc-3cdd2ff8f862 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:06.010486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69f59a-c166-4435-addc-3cdd2ff8f862 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:06.010631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69f59a-c166-4435-addc-3cdd2ff8f862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:06.037295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-de69f59a-c166-4435-addc-3cdd2ff8f862 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:06.038511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-de69f59a-c166-4435-addc-3cdd2ff8f862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:06.123261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 579/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:06.398972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42928fd7-698a-49bc-b574-a2c3a46b0b9e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:06.470685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:06.507730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42928fd7-698a-49bc-b574-a2c3a46b0b9e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:06.543138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:06.543784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:06.544066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:06.544338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:06.555150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:32:06.559848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:32:06.560928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:32:06.561874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 570/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:32:06.568034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69f59a-c166-4435-addc-3cdd2ff8f862 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:06.625824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:06.678229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69f59a-c166-4435-addc-3cdd2ff8f862 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:06.715455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:06.716071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:06.716354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:06.716623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:06.717296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:06.717597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:06.719247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:06.728741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 580/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:06.739484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:06.869871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d9644-5938-4b0b-bc6d-beeb9b08738e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:06.898221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:06.907156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:06.907801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:06.908076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:06.908347 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] PATH_INFO: `/v3/users/b03c4814e4bb4a3099cf8b244a20a97f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:06.909488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] RBAC: Authorizing `identity:get_user(user_id=b03c4814e4bb4a3099cf8b244a20a97f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:06.910918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:06.912536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 571/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:32:06 2022] GET /identity/v3/users/b03c4814e4bb4a3099cf8b244a20a97f => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:06.918601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:07.029280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64b5e4-9802-424d-a0b8-888fa927487f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:07.043665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:07.065429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75fdc3-8444-4394-b796-f2b1f2dbbb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:07.066391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75fdc3-8444-4394-b796-f2b1f2dbbb6b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:07.066667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75fdc3-8444-4394-b796-f2b1f2dbbb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:07.066944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75fdc3-8444-4394-b796-f2b1f2dbbb6b None admin] PATH_INFO: `/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:07.068227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75fdc3-8444-4394-b796-f2b1f2dbbb6b None admin] RBAC: Authorizing `identity:get_project(project_id=adaa4d973454487ea7dbc338bc8e8bfa)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:07.069674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75fdc3-8444-4394-b796-f2b1f2dbbb6b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:07.071368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 581/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:32:06 2022] GET /identity/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:07.080946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:07.191303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbc759-653f-4d0d-aeeb-fa3b57c5177a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:07.231683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7574-4901-4b7d-8cda-d31f8cec3231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:07.232307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7574-4901-4b7d-8cda-d31f8cec3231 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:07.232581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7574-4901-4b7d-8cda-d31f8cec3231 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:07.232855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7574-4901-4b7d-8cda-d31f8cec3231 None admin] PATH_INFO: `/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa/users/b03c4814e4bb4a3099cf8b244a20a97f/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:07.234939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7574-4901-4b7d-8cda-d31f8cec3231 None admin] RBAC: Authorizing `identity:create_grant(project_id=adaa4d973454487ea7dbc338bc8e8bfa, user_id=b03c4814e4bb4a3099cf8b244a20a97f, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:07.236404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7574-4901-4b7d-8cda-d31f8cec3231 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:07.251713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 572/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:32:07 2022] PUT /identity/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa/users/b03c4814e4bb4a3099cf8b244a20a97f/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:32:09.114244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca59c5-596a-4c5d-8583-40a8ac892808 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:09.115125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca59c5-596a-4c5d-8583-40a8ac892808 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:09.115125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca59c5-596a-4c5d-8583-40a8ac892808 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:09.117368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 582/1154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:09.123726 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bb474-c164-4ed8-858b-0de8bcee2e17 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:09.124554 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bb474-c164-4ed8-858b-0de8bcee2e17 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:09.124687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bb474-c164-4ed8-858b-0de8bcee2e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:09.145191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-986bb474-c164-4ed8-858b-0de8bcee2e17 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:09.146391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-986bb474-c164-4ed8-858b-0de8bcee2e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:09.202778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:09.222608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 573/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:09.485965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca59c5-596a-4c5d-8583-40a8ac892808 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:09.596095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca59c5-596a-4c5d-8583-40a8ac892808 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:09.634235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:09.635243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:09.635741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:09.636194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:09.650513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:32:09.655224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:32:09.656457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:32:09.657324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 583/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:32:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:32:09.663274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bb474-c164-4ed8-858b-0de8bcee2e17 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:09.773758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bb474-c164-4ed8-858b-0de8bcee2e17 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:09.809349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:09.809958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:09.810220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:09.810494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:09.811160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:09.811496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:09.812968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:09.821202 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 574/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:32:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:09.827230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:09.873669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:09.933846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f591a8-71c3-4497-b77c-115118c43c90 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:09.981936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:09.982557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:09.982837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:09.983107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] PATH_INFO: `/v3/users/b03c4814e4bb4a3099cf8b244a20a97f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:09.984824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] RBAC: Authorizing `identity:get_user(user_id=b03c4814e4bb4a3099cf8b244a20a97f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:09.986258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:09.987884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 584/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:32:09 2022] GET /identity/v3/users/b03c4814e4bb4a3099cf8b244a20a97f => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:09.994679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:10.049276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:10.104394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78d9c1-03ea-4806-ae15-c23032e52d65 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:10.139821 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:10.140453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:10.140710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:10.140989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:10.212887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:10.214348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:10.272736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:10.336683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 575/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:32:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:32:10.342831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:10.447346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfa16b-1c52-4f61-8805-b778ce946d6c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:10.458363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:10.486396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f4556-5c95-408d-bc88-d7e06bf9258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:10.487024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f4556-5c95-408d-bc88-d7e06bf9258d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:10.487343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f4556-5c95-408d-bc88-d7e06bf9258d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:10.487626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f4556-5c95-408d-bc88-d7e06bf9258d None admin] PATH_INFO: `/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:10.488864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f4556-5c95-408d-bc88-d7e06bf9258d None admin] RBAC: Authorizing `identity:get_project(project_id=adaa4d973454487ea7dbc338bc8e8bfa)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:10.490307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f4556-5c95-408d-bc88-d7e06bf9258d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:10.491994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 585/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:32:10 2022] GET /identity/v3/projects/adaa4d973454487ea7dbc338bc8e8bfa => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:10.498453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:10.601011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89214e-9891-4e05-bafd-57ee6ba932c6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:10.636446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16d02a-9ca4-44a8-87b0-dffaf3152152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:10.637045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16d02a-9ca4-44a8-87b0-dffaf3152152 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:10.637331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16d02a-9ca4-44a8-87b0-dffaf3152152 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:10.637603 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16d02a-9ca4-44a8-87b0-dffaf3152152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:10.638313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16d02a-9ca4-44a8-87b0-dffaf3152152 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=adaa4d973454487ea7dbc338bc8e8bfa, user.id=b03c4814e4bb4a3099cf8b244a20a97f) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:10.638643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16d02a-9ca4-44a8-87b0-dffaf3152152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:10.640391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16d02a-9ca4-44a8-87b0-dffaf3152152 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:10.647030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 576/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:32:10 2022] GET /identity/v3/role_assignments?user.id=b03c4814e4bb4a3099cf8b244a20a97f&scope.project.id=adaa4d973454487ea7dbc338bc8e8bfa&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:12.555052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5f93-874a-4bc1-87b1-ca85122f6b3b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:12.556114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5f93-874a-4bc1-87b1-ca85122f6b3b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:12.556114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5f93-874a-4bc1-87b1-ca85122f6b3b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:12.558140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 586/1162] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:12.567619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc269f5-c871-4b90-a0c5-83a798c63136 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:12.568039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc269f5-c871-4b90-a0c5-83a798c63136 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:12.568245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc269f5-c871-4b90-a0c5-83a798c63136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:12.591092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8dc269f5-c871-4b90-a0c5-83a798c63136 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:12.592325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8dc269f5-c871-4b90-a0c5-83a798c63136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:12.599722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:12.662130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 577/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:12.926888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae5f93-874a-4bc1-87b1-ca85122f6b3b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:13.049217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae5f93-874a-4bc1-87b1-ca85122f6b3b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:13.085006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:13.085610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:13.085891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:13.086170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:13.141519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:13.165951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:13.167456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:13.280802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 587/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:32:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:32:13.290462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc269f5-c871-4b90-a0c5-83a798c63136 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:13.398230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc269f5-c871-4b90-a0c5-83a798c63136 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:13.436806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfea09-5254-4cc9-9906-8102354d7362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:13.437256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfea09-5254-4cc9-9906-8102354d7362 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:13.437539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfea09-5254-4cc9-9906-8102354d7362 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:13.437806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfea09-5254-4cc9-9906-8102354d7362 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:13.445021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfea09-5254-4cc9-9906-8102354d7362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:13.446481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfea09-5254-4cc9-9906-8102354d7362 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f8bfea09-5254-4cc9-9906-8102354d7362 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:32:13.457967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:32:13.459278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:32:13.460777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 578/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:32:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:32:13.468775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:13.553156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:13.576971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c82e15-72b9-4bcf-8673-0f830bbad12c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:13.613119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbb6f5-31ca-49a5-87ad-b3ff308ef16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:13.614056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbb6f5-31ca-49a5-87ad-b3ff308ef16d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:13.614494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbb6f5-31ca-49a5-87ad-b3ff308ef16d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:13.614922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbb6f5-31ca-49a5-87ad-b3ff308ef16d None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:13.616016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbb6f5-31ca-49a5-87ad-b3ff308ef16d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:13.616495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbb6f5-31ca-49a5-87ad-b3ff308ef16d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:13.618726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbb6f5-31ca-49a5-87ad-b3ff308ef16d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:13.628671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 588/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Aug 24 06:32:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:15.537715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5a42b-d5f5-4a08-aaca-ba7f842f3395 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:15.538281 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5a42b-d5f5-4a08-aaca-ba7f842f3395 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:15.538281 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5a42b-d5f5-4a08-aaca-ba7f842f3395 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:15.539669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 579/1167] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:15.545795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9852f-9711-4f9d-9648-c9348d8d7e37 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:15.546494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9852f-9711-4f9d-9648-c9348d8d7e37 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:15.546494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9852f-9711-4f9d-9648-c9348d8d7e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:15.576295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a3c9852f-9711-4f9d-9648-c9348d8d7e37 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:15.578105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a3c9852f-9711-4f9d-9648-c9348d8d7e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:15.650358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 589/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:15.922368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5a42b-d5f5-4a08-aaca-ba7f842f3395 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:15.926520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:16.038917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5a42b-d5f5-4a08-aaca-ba7f842f3395 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:16.073896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4e0cc-1c7b-4c7b-bbcb-d980390ef514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:16.074491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4e0cc-1c7b-4c7b-bbcb-d980390ef514 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:16.074774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4e0cc-1c7b-4c7b-bbcb-d980390ef514 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:16.075055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4e0cc-1c7b-4c7b-bbcb-d980390ef514 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:16.075752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4e0cc-1c7b-4c7b-bbcb-d980390ef514 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:16.077138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4e0cc-1c7b-4c7b-bbcb-d980390ef514 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:16.097511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 580/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Aug 24 06:32:15 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:32:18.038044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10b905-9bd9-45ef-b717-6ce56083d38f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:18.038730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10b905-9bd9-45ef-b717-6ce56083d38f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:18.038730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10b905-9bd9-45ef-b717-6ce56083d38f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:18.040038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 590/1170] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:18.046806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e713199-da50-4093-92b9-9e806a43eff1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:18.047571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e713199-da50-4093-92b9-9e806a43eff1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:18.047727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e713199-da50-4093-92b9-9e806a43eff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:18.072087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5e713199-da50-4093-92b9-9e806a43eff1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:18.073280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5e713199-da50-4093-92b9-9e806a43eff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:18.130365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:18.142387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 581/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:18.405718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10b905-9bd9-45ef-b717-6ce56083d38f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:18.497969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:18.529825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10b905-9bd9-45ef-b717-6ce56083d38f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:18.574783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38105143-cca8-492b-bec5-3f966b3b2eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:18.575772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38105143-cca8-492b-bec5-3f966b3b2eda None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:18.576186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38105143-cca8-492b-bec5-3f966b3b2eda None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:18.576581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38105143-cca8-492b-bec5-3f966b3b2eda None admin] PATH_INFO: `/v3/domains/5ee5d32b7d094a86ab50c5242c6d9731` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:18.593148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38105143-cca8-492b-bec5-3f966b3b2eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=5ee5d32b7d094a86ab50c5242c6d9731)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:18.595035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38105143-cca8-492b-bec5-3f966b3b2eda None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:18.597487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 591/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Aug 24 06:32:18 2022] GET /identity/v3/domains/5ee5d32b7d094a86ab50c5242c6d9731 => generated 267 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:18.606312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e713199-da50-4093-92b9-9e806a43eff1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:18.721040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e713199-da50-4093-92b9-9e806a43eff1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:18.758279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9f2dc-397a-445a-81bb-fe2ca714f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:18.758882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f2dc-397a-445a-81bb-fe2ca714f0b3 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:18.759164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f2dc-397a-445a-81bb-fe2ca714f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:18.759464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9f2dc-397a-445a-81bb-fe2ca714f0b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:18.760118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9f2dc-397a-445a-81bb-fe2ca714f0b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:18.761551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9f2dc-397a-445a-81bb-fe2ca714f0b3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:18.791862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 582/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:32:18 2022] POST /identity/v3/projects => generated 361 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:32:20.727485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b25760-c119-4bd9-8018-fd9ec8451fc2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:20.728056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b25760-c119-4bd9-8018-fd9ec8451fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:20.728056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b25760-c119-4bd9-8018-fd9ec8451fc2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:20.729427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 592/1174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:20.736176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd274-0e5d-4411-9fa3-67b805994cec None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:20.736908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd274-0e5d-4411-9fa3-67b805994cec None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:20.737028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd274-0e5d-4411-9fa3-67b805994cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:20.766578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-76afd274-0e5d-4411-9fa3-67b805994cec None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:20.767820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-76afd274-0e5d-4411-9fa3-67b805994cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:20.814349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:20.847937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 583/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:21.114108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b25760-c119-4bd9-8018-fd9ec8451fc2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:21.230419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b25760-c119-4bd9-8018-fd9ec8451fc2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:21.260924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:21.268249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48e315-fb17-4e14-b87d-37187e8643f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:21.268886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e315-fb17-4e14-b87d-37187e8643f5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:21.269169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e315-fb17-4e14-b87d-37187e8643f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:21.269444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48e315-fb17-4e14-b87d-37187e8643f5 None admin] PATH_INFO: `/v3/domains/5ee5d32b7d094a86ab50c5242c6d9731` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:21.270576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48e315-fb17-4e14-b87d-37187e8643f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5ee5d32b7d094a86ab50c5242c6d9731)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:21.272051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48e315-fb17-4e14-b87d-37187e8643f5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:21.273631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 593/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Aug 24 06:32:21 2022] GET /identity/v3/domains/5ee5d32b7d094a86ab50c5242c6d9731 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:21.282211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd274-0e5d-4411-9fa3-67b805994cec None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:21.395407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afd274-0e5d-4411-9fa3-67b805994cec None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:21.432408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a734d9b-d893-4490-b9d5-4b20575fe48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:21.433033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a734d9b-d893-4490-b9d5-4b20575fe48b None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:21.433315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a734d9b-d893-4490-b9d5-4b20575fe48b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:21.433590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a734d9b-d893-4490-b9d5-4b20575fe48b None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:21.434242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a734d9b-d893-4490-b9d5-4b20575fe48b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:21.435738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a734d9b-d893-4490-b9d5-4b20575fe48b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:21.466068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 584/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:32:21 2022] POST /identity/v3/users => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:32:23.401306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f887-6075-4a10-95a3-fc7095bb106d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:23.401874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f887-6075-4a10-95a3-fc7095bb106d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:23.401874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f887-6075-4a10-95a3-fc7095bb106d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:23.403210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 594/1178] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:23.409674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affa9a0-a499-4643-a363-4ef13aebf412 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:23.410421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affa9a0-a499-4643-a363-4ef13aebf412 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:23.410570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affa9a0-a499-4643-a363-4ef13aebf412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:23.439905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5affa9a0-a499-4643-a363-4ef13aebf412 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:23.441470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5affa9a0-a499-4643-a363-4ef13aebf412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:23.465598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:23.513568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 585/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:23.776335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58f887-6075-4a10-95a3-fc7095bb106d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:23.909300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58f887-6075-4a10-95a3-fc7095bb106d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:23.954708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:23.955365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:23.955647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:23.955924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:23.967048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:32:23.972217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:32:23.973316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:32:23.974420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 595/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:32:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:32:23.980369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affa9a0-a499-4643-a363-4ef13aebf412 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:24.085793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affa9a0-a499-4643-a363-4ef13aebf412 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:24.121908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:24.122510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:24.122790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:24.123063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:24.123765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:24.124066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:24.125531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:24.133850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 586/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:32:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:24.140070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:24.209536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:24.252056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe55bec-5e3f-4954-8b66-e55ba7b84e51 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:24.286981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:24.287636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:24.287911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:24.288185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] PATH_INFO: `/v3/users/7091a14cff2141cbb231ca5f52273999` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:24.304261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] RBAC: Authorizing `identity:get_user(user_id=7091a14cff2141cbb231ca5f52273999)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:24.305712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:24.307380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 596/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:32:24 2022] GET /identity/v3/users/7091a14cff2141cbb231ca5f52273999 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:24.314092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:24.340382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:24.421292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed08f3-3d6c-408a-817c-169a9831a917 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:24.456986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:24.457598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:24.457873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:24.458148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:24.529447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:24.531015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:24.554959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:24.634912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 587/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:32:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:32:24.641286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:24.758337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b981798-9fbc-4fd5-b93a-63a3caabb023 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:24.777562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:24.796299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe7715-c6b4-4c6a-b9cc-18c93e123c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:24.797048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe7715-c6b4-4c6a-b9cc-18c93e123c8c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:24.797330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe7715-c6b4-4c6a-b9cc-18c93e123c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:24.797601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe7715-c6b4-4c6a-b9cc-18c93e123c8c None admin] PATH_INFO: `/v3/projects/5dcfdb754b8c416d86b36b133f312af0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:24.799056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe7715-c6b4-4c6a-b9cc-18c93e123c8c None admin] RBAC: Authorizing `identity:get_project(project_id=5dcfdb754b8c416d86b36b133f312af0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:24.800714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe7715-c6b4-4c6a-b9cc-18c93e123c8c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:24.802343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 597/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:32:24 2022] GET /identity/v3/projects/5dcfdb754b8c416d86b36b133f312af0 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:32:24.811748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:24.911939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b731a-bbdd-455f-9724-4c6147c86a99 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:24.935316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:24.948876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9526e-ad59-4d11-a34b-071ce446569f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:24.949476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9526e-ad59-4d11-a34b-071ce446569f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:24.949755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9526e-ad59-4d11-a34b-071ce446569f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:24.950024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9526e-ad59-4d11-a34b-071ce446569f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:24.950753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9526e-ad59-4d11-a34b-071ce446569f None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=5dcfdb754b8c416d86b36b133f312af0, user.id=7091a14cff2141cbb231ca5f52273999) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:24.951057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9526e-ad59-4d11-a34b-071ce446569f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:24.952520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9526e-ad59-4d11-a34b-071ce446569f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:24.958075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 588/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:32:24 2022] GET /identity/v3/role_assignments?user.id=7091a14cff2141cbb231ca5f52273999&scope.project.id=5dcfdb754b8c416d86b36b133f312af0&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:26.825192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23bb4-cc7b-408d-99c5-7cfb8efe2500 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:26.826159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23bb4-cc7b-408d-99c5-7cfb8efe2500 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:26.826159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23bb4-cc7b-408d-99c5-7cfb8efe2500 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:26.828131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 598/1186] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:26.834774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175eb9-313f-4aa8-b121-c042d8e134db None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:26.835230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175eb9-313f-4aa8-b121-c042d8e134db None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:26.835491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175eb9-313f-4aa8-b121-c042d8e134db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:26.863294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9c175eb9-313f-4aa8-b121-c042d8e134db None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:26.864503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9c175eb9-313f-4aa8-b121-c042d8e134db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:26.938363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 589/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:27.212378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23bb4-cc7b-408d-99c5-7cfb8efe2500 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:27.323098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23bb4-cc7b-408d-99c5-7cfb8efe2500 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:27.359363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:27.359989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:27.360262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:27.360532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:27.372150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:32:27.376772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:32:27.377842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:32:27.379006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 599/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:32:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:32:27.387716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175eb9-313f-4aa8-b121-c042d8e134db None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:27.496219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175eb9-313f-4aa8-b121-c042d8e134db None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:27.515697 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:27.535899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:27.536507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:27.536796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:27.537077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:27.537753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:27.538061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:27.539485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:27.548196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 590/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:32:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:27.555253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:27.610484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:27.676875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a2b45-2759-4f99-8b74-623524c1eaa4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:27.712984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:27.713601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:27.713883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:27.714163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] PATH_INFO: `/v3/users/7091a14cff2141cbb231ca5f52273999` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:27.715363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] RBAC: Authorizing `identity:get_user(user_id=7091a14cff2141cbb231ca5f52273999)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:27.716794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:27.718591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 600/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:32:27 2022] GET /identity/v3/users/7091a14cff2141cbb231ca5f52273999 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:27.728057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:27.838592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad6b9c-82fb-408a-9be7-59dee97c94a6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:27.880429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81487843-50ad-4cf8-b545-2b2dd980aab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:27.881059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81487843-50ad-4cf8-b545-2b2dd980aab9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:27.881340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81487843-50ad-4cf8-b545-2b2dd980aab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:27.881615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81487843-50ad-4cf8-b545-2b2dd980aab9 None admin] PATH_INFO: `/v3/projects/5dcfdb754b8c416d86b36b133f312af0` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:27.882788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81487843-50ad-4cf8-b545-2b2dd980aab9 None admin] RBAC: Authorizing `identity:get_project(project_id=5dcfdb754b8c416d86b36b133f312af0)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:27.884267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81487843-50ad-4cf8-b545-2b2dd980aab9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:27.885915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 591/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:32:27 2022] GET /identity/v3/projects/5dcfdb754b8c416d86b36b133f312af0 => generated 361 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:27.894793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:28.016427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe841ae-c1c4-4db1-9b0d-200a9ff0a18b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:28.032343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:28.057681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6133f62-879b-4978-a45c-1919635e9909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:28.058335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6133f62-879b-4978-a45c-1919635e9909 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:28.058620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6133f62-879b-4978-a45c-1919635e9909 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:28.058895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6133f62-879b-4978-a45c-1919635e9909 None admin] PATH_INFO: `/v3/projects/5dcfdb754b8c416d86b36b133f312af0/users/7091a14cff2141cbb231ca5f52273999/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:28.061112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6133f62-879b-4978-a45c-1919635e9909 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dcfdb754b8c416d86b36b133f312af0, user_id=7091a14cff2141cbb231ca5f52273999, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:28.062735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6133f62-879b-4978-a45c-1919635e9909 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:28.078516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 601/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:32:27 2022] PUT /identity/v3/projects/5dcfdb754b8c416d86b36b133f312af0/users/7091a14cff2141cbb231ca5f52273999/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:32:29.958542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685a81-a805-4fbb-809d-edcb5294b83a None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:29.959463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685a81-a805-4fbb-809d-edcb5294b83a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:29.959463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685a81-a805-4fbb-809d-edcb5294b83a None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:29.961085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 592/1193] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:29.969101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f19b-769e-41fd-b061-ec560042645b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:29.969549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f19b-769e-41fd-b061-ec560042645b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:29.969648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f19b-769e-41fd-b061-ec560042645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:29.988871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2e28f19b-769e-41fd-b061-ec560042645b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:29.990061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2e28f19b-769e-41fd-b061-ec560042645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:30.059941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 602/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:32:30.329593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5685a81-a805-4fbb-809d-edcb5294b83a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:30.400736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:30.461921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5685a81-a805-4fbb-809d-edcb5294b83a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:30.502122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:30.502747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:30.503023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:30.503328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:30.513762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:32:30.518463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:32:30.519625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:32:30.520673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 593/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:32:30.528613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28f19b-769e-41fd-b061-ec560042645b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:30.643903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28f19b-769e-41fd-b061-ec560042645b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:30.646230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:30.687844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:30.688458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:30.688733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:30.689018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:30.689694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:30.690002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:30.691771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:30.699705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 603/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:30.706024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:30.818098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72748777-948b-405b-8978-7ade60d52ff7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:30.824958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:30.854119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:30.854746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:30.855028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:30.855335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] PATH_INFO: `/v3/users/7091a14cff2141cbb231ca5f52273999` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:30.856506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] RBAC: Authorizing `identity:get_user(user_id=7091a14cff2141cbb231ca5f52273999)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:30.857939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:30.859890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 594/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Aug 24 06:32:30 2022] GET /identity/v3/users/7091a14cff2141cbb231ca5f52273999 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:30.866624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:30.991801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0e5d6-7db2-405d-8ecc-a7414fa9ebb5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:31.035799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:31.036407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:31.036689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:31.036998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:31.074137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:31.117854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:31.119361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:31.227246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 604/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:32:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:32:31.233863 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:31.338132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcde14-6447-4e1a-81b3-63f217af73f1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:31.371306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066a3cc-77f0-4859-b82e-707868fa44c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:31.371923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066a3cc-77f0-4859-b82e-707868fa44c4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:31.372199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066a3cc-77f0-4859-b82e-707868fa44c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:31.372479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066a3cc-77f0-4859-b82e-707868fa44c4 None admin] PATH_INFO: `/v3/projects/5dcfdb754b8c416d86b36b133f312af0` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:31.373717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066a3cc-77f0-4859-b82e-707868fa44c4 None admin] RBAC: Authorizing `identity:get_project(project_id=5dcfdb754b8c416d86b36b133f312af0)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:31.375497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066a3cc-77f0-4859-b82e-707868fa44c4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:31.377137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 595/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Aug 24 06:32:31 2022] GET /identity/v3/projects/5dcfdb754b8c416d86b36b133f312af0 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:31.384230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:31.455164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:31.492340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7adb00-f86f-42c0-8302-33f42dd3c205 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:31.529764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6594d-eab8-4db5-b599-6c2521413578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:31.529764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6594d-eab8-4db5-b599-6c2521413578 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:31.529764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6594d-eab8-4db5-b599-6c2521413578 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:31.529764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6594d-eab8-4db5-b599-6c2521413578 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:31.530129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6594d-eab8-4db5-b599-6c2521413578 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=5dcfdb754b8c416d86b36b133f312af0, user.id=7091a14cff2141cbb231ca5f52273999) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:32:31.530129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6594d-eab8-4db5-b599-6c2521413578 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:31.531289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6594d-eab8-4db5-b599-6c2521413578 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:31.537453 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 605/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:32:31 2022] GET /identity/v3/role_assignments?user.id=7091a14cff2141cbb231ca5f52273999&scope.project.id=5dcfdb754b8c416d86b36b133f312af0&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:32:41.366628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=79 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:41.534579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=35 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:28.058798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845b925-c05f-4c58-b38c-54a084073e25 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:28.059521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845b925-c05f-4c58-b38c-54a084073e25 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:28.059699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845b925-c05f-4c58-b38c-54a084073e25 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:28.062051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 596/1201] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:28.068072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c536a-7192-406f-82b5-1762d18fa820 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:28.068601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c536a-7192-406f-82b5-1762d18fa820 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:28.068699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c536a-7192-406f-82b5-1762d18fa820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:28.090245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f05c536a-7192-406f-82b5-1762d18fa820 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:28.091764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f05c536a-7192-406f-82b5-1762d18fa820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:28.186783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 606/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:28.504167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845b925-c05f-4c58-b38c-54a084073e25 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:28.648761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845b925-c05f-4c58-b38c-54a084073e25 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:28.701345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:28.702011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:28.702533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:28.702900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:28.719857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:28.721938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:28.755422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:28.757532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:28.759383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:28.759383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:28.759383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:28.759383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 597/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:28.763673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c536a-7192-406f-82b5-1762d18fa820 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:28.911704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c536a-7192-406f-82b5-1762d18fa820 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:28.947394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:28.953784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5953efa-cb8b-4ffc-a36f-e441f8d05981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:28.954599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953efa-cb8b-4ffc-a36f-e441f8d05981 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:28.954858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953efa-cb8b-4ffc-a36f-e441f8d05981 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:28.955343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5953efa-cb8b-4ffc-a36f-e441f8d05981 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:28.956045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5953efa-cb8b-4ffc-a36f-e441f8d05981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:28.956359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5953efa-cb8b-4ffc-a36f-e441f8d05981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:28.957759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5953efa-cb8b-4ffc-a36f-e441f8d05981 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:28.972255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 607/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:28.978889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:29.048341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:29.102278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda40cd9-4906-4b2c-9859-c82f82cb33b4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:29.148761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384d3d6-c278-4815-b17c-2ab5dd380080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:29.149412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384d3d6-c278-4815-b17c-2ab5dd380080 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:29.149808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384d3d6-c278-4815-b17c-2ab5dd380080 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:29.150238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384d3d6-c278-4815-b17c-2ab5dd380080 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:29.151008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384d3d6-c278-4815-b17c-2ab5dd380080 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:29.152768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384d3d6-c278-4815-b17c-2ab5dd380080 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:29.180242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 598/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Aug 24 06:33:28 2022] POST /identity/v3/users => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:33:31.459542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92eaa-eb0e-4b4f-ac3a-c6713d15d058 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:31.460131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92eaa-eb0e-4b4f-ac3a-c6713d15d058 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:31.460131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e92eaa-eb0e-4b4f-ac3a-c6713d15d058 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:31.461460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 608/1206] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:31.468704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9b07a-c6b8-4424-835c-cf52f1051f9a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:31.468704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9b07a-c6b8-4424-835c-cf52f1051f9a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:31.468704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9b07a-c6b8-4424-835c-cf52f1051f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:31.494803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-19a9b07a-c6b8-4424-835c-cf52f1051f9a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:31.496585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-19a9b07a-c6b8-4424-835c-cf52f1051f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:31.589359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 599/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:31.883877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92eaa-eb0e-4b4f-ac3a-c6713d15d058 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:32.017007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e92eaa-eb0e-4b4f-ac3a-c6713d15d058 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:32.060590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:32.061275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:32.061557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:32.061858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:32.075959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:33:32.081077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:32.082200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:32.083152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 609/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:33:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:32.088513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9b07a-c6b8-4424-835c-cf52f1051f9a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:32.133634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:32.207005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9b07a-c6b8-4424-835c-cf52f1051f9a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:32.244397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:32.245036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:32.245299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:32.245569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:32.246243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:32.246553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:32.247962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:32.257353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 600/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:32.263493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:32.323786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:32.381593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ecb87-b13f-43b4-989c-a1121b7e5332 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:32.418984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:32.419776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:32.420056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:32.420327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:32.434935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:32.436494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:32.452021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:32.453982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:32.455150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:32.455150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:32.455150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:32.455150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 610/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:32.459953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:32.574689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:32.577565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b0763-89b6-4afb-a0b5-98fd8c7c0f91 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:32.617615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:32.618517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:32.618798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:32.619070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:32.619874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:32.620179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:32.621866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:32.636327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 601/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:32.642455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:32.757019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635c7bf-8852-46d8-81a6-ca3ee2071929 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:32.758611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:32.796131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:32.796802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:32.797052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:32.797328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:32.807053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:32.808754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:33:32.822948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:32.824306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:32.825310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 611/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:32.830519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:32.947752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd41dc-d046-4197-bbf1-4bc8673f3f2c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:32.988192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:32.989787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:32.989787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:32.989787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:32.990186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:32.990502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:32.991969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:33.015731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 602/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 06:33:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:33.022462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:33.162315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b444-9c57-4786-8782-8320ec5f2274 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:33.203629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:33.204883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:33.205196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:33.205473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:33.243086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:33.287872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:33.289271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:33.403860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 612/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:33:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:33:33.410140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:33.430035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:33.516006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d213b-a14b-44f6-96ef-6071f0c0f519 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:33.553824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:33.554391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:33.554675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:33.554992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:33.566195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:33.568049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:33.582852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:33.584370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:33.585375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:33.585375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:33.585375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:33.585375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 603/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:33:33.591764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:33.673951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:33.724158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59dabe-34ee-49ff-a50e-94abba2b3f31 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:33.761895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:33.762518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:33.762800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:33.763072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:33.763765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:33.764073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:33.765463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:33.777628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 613/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:33.783446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:33.859008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:33.892104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ec63d-f6ef-4b3e-af97-0d81b8d5ae23 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:33.926719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:33.927405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:33.927691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:33.927966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:33.935359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:33.936816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:33:33.949003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:33.950133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:33.951357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:33.955291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 604/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:33:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:33.956327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:34.064941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16943832-8e84-43a5-be9f-809284e85f0e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:34.093704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:34.110584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93270f56-2cc5-47e1-86de-941dde7702fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:34.111234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93270f56-2cc5-47e1-86de-941dde7702fc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:34.111534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93270f56-2cc5-47e1-86de-941dde7702fc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:34.111811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93270f56-2cc5-47e1-86de-941dde7702fc None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:34.112516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93270f56-2cc5-47e1-86de-941dde7702fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:34.112862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93270f56-2cc5-47e1-86de-941dde7702fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:34.114350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93270f56-2cc5-47e1-86de-941dde7702fc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:34.128366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 614/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:33:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:34.134761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:34.254203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3113724-5e6d-4fe7-8fb1-fba3ff29c098 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:34.286607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:34.294299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f7551-f7df-4ce2-8dab-bbd7eb9f826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:34.294988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7551-f7df-4ce2-8dab-bbd7eb9f826c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:34.295304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7551-f7df-4ce2-8dab-bbd7eb9f826c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:34.295579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7551-f7df-4ce2-8dab-bbd7eb9f826c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:34.296326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f7551-f7df-4ce2-8dab-bbd7eb9f826c None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=d115d3f3adc54a17bf80962a6ae3ab82) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:34.296639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f7551-f7df-4ce2-8dab-bbd7eb9f826c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:34.299996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f7551-f7df-4ce2-8dab-bbd7eb9f826c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:34.306491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 605/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:33:34 2022] GET /identity/v3/role_assignments?user.id=d115d3f3adc54a17bf80962a6ae3ab82&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:36.203032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccedd8d-65e3-4e8c-bc44-50042b3b92b2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:36.203648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccedd8d-65e3-4e8c-bc44-50042b3b92b2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:36.203648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccedd8d-65e3-4e8c-bc44-50042b3b92b2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:36.205271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 615/1220] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:36.211103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63017-2675-4264-a340-50d275b54618 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:36.211764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63017-2675-4264-a340-50d275b54618 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:36.211764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63017-2675-4264-a340-50d275b54618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:36.232679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c3e63017-2675-4264-a340-50d275b54618 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:36.233879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c3e63017-2675-4264-a340-50d275b54618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:36.315417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 606/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:36.588797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccedd8d-65e3-4e8c-bc44-50042b3b92b2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:36.704464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccedd8d-65e3-4e8c-bc44-50042b3b92b2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:36.755218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:36.755955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:36.756274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:36.756587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:36.768159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:33:36.774021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:36.775263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:36.776286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 616/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:33:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:36.781919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e63017-2675-4264-a340-50d275b54618 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:36.905654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e63017-2675-4264-a340-50d275b54618 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:36.933914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:36.946702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:36.947377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:36.947663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:36.947938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:36.948615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:36.948924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:36.950294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:36.959052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 607/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:33:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:36.965582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:37.014256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:37.076044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50038f58-ac39-4a9a-9377-51a9b88ae7cd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:37.117258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:37.117902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:37.118189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:37.118466 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:37.129446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:37.130973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:37.149987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:37.152272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:37.154929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:37.154929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:37.154929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:37.154929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 617/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:33:37.160909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:37.277825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1c458-b3b2-41d0-b918-784cb4f18181 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:37.315776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:37.316394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:37.317038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:37.317314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:37.317961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:37.318265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:37.319687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:37.332498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 608/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:37.338504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:37.436783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:37.444894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec30277-2334-4c7b-aa0f-ade4e62fa2d5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:37.480825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:37.481480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:37.481759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:37.482031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:37.489681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:37.491258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:33:37.502235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:37.503430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:37.504439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 618/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:33:37.509485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:37.562975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:37.622232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eeace-a857-4561-97a0-8a0aa86b01f7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:37.661769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:37.662484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:37.662882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:37.663322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:37.664181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:37.664622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:37.666409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:37.687917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 609/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 06:33:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:37.694993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:37.814786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491ad67-fe41-4773-8e43-4a8f2b47bb1e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:37.851825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:37.852465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:37.852749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:37.853028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:37.864611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:37.866070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:37.881326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:37.882522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:37.883546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:37.883546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:37.883546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:37.883546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 619/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:37.888499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:38.004335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082050db-5a45-4273-9e93-c0fb9aa044d9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:38.006678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:38.041254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:38.041902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:38.042191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:38.042471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:38.043124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:38.043466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:38.044844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:38.057329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 610/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:38.063590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:38.074684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:38.189369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3243629-cb4d-428a-827a-79a7ffc2154c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:38.228465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:38.229177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:38.229457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:38.229787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:38.239613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:38.241376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:33:38.255105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:38.256184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:38.257197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 620/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:33:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:38.262625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:38.371905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d14df-1aae-411d-8a2e-824415d0b626 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:38.408159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ba0ba-a4cd-4e8a-a3e7-cb121b33c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:38.408799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba0ba-a4cd-4e8a-a3e7-cb121b33c743 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:38.409081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba0ba-a4cd-4e8a-a3e7-cb121b33c743 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:38.409365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba0ba-a4cd-4e8a-a3e7-cb121b33c743 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:38.410091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba0ba-a4cd-4e8a-a3e7-cb121b33c743 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:38.410366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba0ba-a4cd-4e8a-a3e7-cb121b33c743 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:38.411831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba0ba-a4cd-4e8a-a3e7-cb121b33c743 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:38.425318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 611/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:33:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:38.431391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:38.502517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:38.539293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15773c2d-57ba-415e-b0a9-e761760b2785 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:38.576481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2e053-b49d-436e-8db6-044d53c7b353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:38.577157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2e053-b49d-436e-8db6-044d53c7b353 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:38.577443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2e053-b49d-436e-8db6-044d53c7b353 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:38.577722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2e053-b49d-436e-8db6-044d53c7b353 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/d115d3f3adc54a17bf80962a6ae3ab82/roles/72fda378772c45f4b41c5f973cc7cda2` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:38.593556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2e053-b49d-436e-8db6-044d53c7b353 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=d115d3f3adc54a17bf80962a6ae3ab82, role_id=72fda378772c45f4b41c5f973cc7cda2)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:38.595434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2e053-b49d-436e-8db6-044d53c7b353 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:38.613652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 621/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:33:38 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/d115d3f3adc54a17bf80962a6ae3ab82/roles/72fda378772c45f4b41c5f973cc7cda2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:33:40.886886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6330682-f129-4595-b5d3-9fb8037299d3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:40.887483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6330682-f129-4595-b5d3-9fb8037299d3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:40.887483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6330682-f129-4595-b5d3-9fb8037299d3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:40.888822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 612/1233] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:40.894242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76ee2-ae5e-456c-b5ca-b0998f1fe16d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:40.894662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76ee2-ae5e-456c-b5ca-b0998f1fe16d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:40.894776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76ee2-ae5e-456c-b5ca-b0998f1fe16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:40.914626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-14d76ee2-ae5e-456c-b5ca-b0998f1fe16d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:40.915961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-14d76ee2-ae5e-456c-b5ca-b0998f1fe16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:40.988615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 622/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:41.254816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6330682-f129-4595-b5d3-9fb8037299d3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:41.282505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:41.365139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6330682-f129-4595-b5d3-9fb8037299d3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:41.399877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:41.400521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.400812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.401062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.412408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:33:41.417072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:41.418353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:41.419530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 613/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Aug 24 06:33:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:41.424588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d76ee2-ae5e-456c-b5ca-b0998f1fe16d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:41.477174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:41.537087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d76ee2-ae5e-456c-b5ca-b0998f1fe16d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:41.575532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:41.576150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.576428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.576700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.577370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:41.577677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:41.579184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:41.587401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 623/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Aug 24 06:33:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:41.593150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:41.707561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:41.726645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4fbe1-c4d5-4bdd-9c5c-74f08efb6a06 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:41.762245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:41.762876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.763154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.763463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.774339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:41.775816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:41.790629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:41.791994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:41.793133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:41.793133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:41.793133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:41.793133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 614/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:41.797784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:41.905389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb80fa-d68b-4f63-b02c-2810c93203fa None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:41.915731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:41.945689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:41.946310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.946597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.946910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.947724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:41.948112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:41.949755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:41.963370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 624/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:41.969239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:42.097466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edd22d-f65c-4334-849d-a6e180afc31f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:42.136055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:42.136757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:42.137135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:42.137406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:42.140319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:42.146978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:42.148596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:33:42.160563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:42.161630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:42.162782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 615/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:33:42.167663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:42.291562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8cd18-7437-476a-97fa-d9d6bc52c242 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:42.336252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:42.336910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:42.337192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:42.337465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:42.338205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:42.338515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:42.340011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:42.358929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 625/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 06:33:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:42.365283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:42.480836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f69bfe-7f15-4d0e-865a-fb5864d3e69b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:42.518621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:42.519394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:42.519772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:42.520042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:42.599567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:42.606461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:42.607954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:42.748419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 616/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:33:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:33:42.749812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:42.779800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:42.862363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cf897-e180-4ac0-ad53-feed122efaf8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:42.902216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:42.902960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:42.903312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:42.903642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:42.915994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:42.917695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:42.933527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:42.934763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:42.936366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:42.936366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:42.936366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:42.936366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 626/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:33:42.943030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:43.066330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:43.075376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dc6e3-fa43-4320-bce7-996ed4f6a482 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:43.124259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:43.125154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:43.125461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:43.125827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:43.126525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:43.126858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:43.128433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:43.143264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 617/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:43.147331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:43.239110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:43.266491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fcb1a-b55d-4cdc-a986-8d52ff16f541 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:43.303579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:43.304300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:43.304675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:43.304989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:43.313069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:43.315075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:33:43.328242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:43.329335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:43.330259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 627/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:43.336175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:43.449785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ce231-adfb-45f0-ba3c-a3eca1dcf8e5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:43.489943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aadf26-1e4f-41f4-8420-3ea91ad7451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:43.490541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aadf26-1e4f-41f4-8420-3ea91ad7451b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:43.490820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aadf26-1e4f-41f4-8420-3ea91ad7451b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:43.491110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aadf26-1e4f-41f4-8420-3ea91ad7451b None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:43.491820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aadf26-1e4f-41f4-8420-3ea91ad7451b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:43.492124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aadf26-1e4f-41f4-8420-3ea91ad7451b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:43.493561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aadf26-1e4f-41f4-8420-3ea91ad7451b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:43.508326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 618/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:43.515629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:43.636535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df76d1-b369-49c1-a198-0019bef78a34 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:43.684320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f231f0-90ff-4df9-8372-fafa2862f3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:43.685276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f231f0-90ff-4df9-8372-fafa2862f3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:43.685673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f231f0-90ff-4df9-8372-fafa2862f3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:43.686026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f231f0-90ff-4df9-8372-fafa2862f3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:43.686963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f231f0-90ff-4df9-8372-fafa2862f3a0 None admin] RBAC: Adding query filter params (role.id=72fda378772c45f4b41c5f973cc7cda2, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=d115d3f3adc54a17bf80962a6ae3ab82) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:43.688962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f231f0-90ff-4df9-8372-fafa2862f3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:43.689503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f231f0-90ff-4df9-8372-fafa2862f3a0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:43.699240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 628/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:33:43 2022] GET /identity/v3/role_assignments?user.id=d115d3f3adc54a17bf80962a6ae3ab82&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=72fda378772c45f4b41c5f973cc7cda2 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:45.712818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2684e-0fad-43b2-89d3-0be7748b4b86 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:45.713451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2684e-0fad-43b2-89d3-0be7748b4b86 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:45.713451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2684e-0fad-43b2-89d3-0be7748b4b86 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:45.714978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 619/1247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:45.721068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbb413-b837-4041-921d-850e2a2374a8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:45.721503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbb413-b837-4041-921d-850e2a2374a8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:45.721806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbb413-b837-4041-921d-850e2a2374a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:45.728956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:45.743621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-03dbb413-b837-4041-921d-850e2a2374a8 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:45.745149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-03dbb413-b837-4041-921d-850e2a2374a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:45.820987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 629/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:46.105766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2684e-0fad-43b2-89d3-0be7748b4b86 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:46.126984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:46.227360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2684e-0fad-43b2-89d3-0be7748b4b86 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:46.265746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:46.266625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:46.266910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:46.267260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:46.278830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:33:46.284385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:46.285615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:46.286631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 620/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:46.292189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbb413-b837-4041-921d-850e2a2374a8 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:46.410473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbb413-b837-4041-921d-850e2a2374a8 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:46.441454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:46.447629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:46.448265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:46.448621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:46.448834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:46.449463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:46.449787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:46.451211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:46.460519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 630/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:46.466033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:46.562192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:46.588043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd8d33-c5f2-4923-90fb-7f312a80f9d9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:46.624655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:46.625297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:46.625581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:46.625999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:46.636411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:46.637903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:46.654699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:46.655838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:46.656763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:46.656763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:46.656763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:46.661283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 621/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:46.662675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:46.786486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c157d-afe0-4b11-bc50-f95456f922d9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:46.827139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:46.827799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:46.828086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:46.828361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:46.829808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:46.830122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:46.831539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:46.849926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 631/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:46.851230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:46.962669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9eb76-ae86-403a-bf18-3a25bf6e56f5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:46.990038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:46.998420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:46.999066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:46.999382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:46.999662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] PATH_INFO: `/v3/users/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:47.009681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:47.011490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:33:47.025840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:47.027020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:47.028331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:47.032943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 622/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:47.034012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:47.104648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:47.153792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc116db-6fce-4b46-9afd-41e20ffb1632 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:47.196593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:47.197614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:47.198013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:47.198300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:47.199219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:47.199546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:47.201071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:47.225480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 632/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 06:33:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:47.231591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:47.340322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520a285-13c8-402b-a1a1-33c133b170da None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:47.382863 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:47.383550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:47.383799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:47.384077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:47.447868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:47.463099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:47.464619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:47.572484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 623/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:33:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:33:47.578437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:47.700867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b14558-6961-4dac-8e96-b5d0e463a7c0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:47.716150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:47.740723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:47.741368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:47.741647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:47.741924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:47.755675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:47.757361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:47.773384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:47.774597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:47.775690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:47.775690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:47.775690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:47.775690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 633/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:47.781307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:47.874149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:47.886623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587b7f6-4e85-4d60-b07a-7e5435c75b18 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:47.925477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:47.926127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:47.926461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:47.926812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:47.927609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:47.927921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:47.929496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:47.942828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 624/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:47.949611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:48.062110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f55c76-3934-4e44-84b4-2874a627b415 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:48.100396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:48.101066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:48.101346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:48.101620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:48.109283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:48.110818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:33:48.123576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:48.124548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:48.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 634/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:48.131483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:48.238645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725245-4617-492e-8c07-6b8293ab16ac None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:48.273787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f75b4-174c-4dcf-a7d4-d52aa781bfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:48.274420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f75b4-174c-4dcf-a7d4-d52aa781bfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:48.274682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f75b4-174c-4dcf-a7d4-d52aa781bfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:48.274945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f75b4-174c-4dcf-a7d4-d52aa781bfe5 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:48.275678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f75b4-174c-4dcf-a7d4-d52aa781bfe5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:48.275995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f75b4-174c-4dcf-a7d4-d52aa781bfe5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:48.277410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f75b4-174c-4dcf-a7d4-d52aa781bfe5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:48.289754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 625/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:33:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:48.296379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:48.325595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:48.410146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8b34-dba1-48bc-b691-6c54525355a3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:48.443921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b4571-fece-4331-b063-a24d05dfb508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:48.444536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b4571-fece-4331-b063-a24d05dfb508 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:48.444811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b4571-fece-4331-b063-a24d05dfb508 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:48.445077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b4571-fece-4331-b063-a24d05dfb508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:48.445825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b4571-fece-4331-b063-a24d05dfb508 None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=d115d3f3adc54a17bf80962a6ae3ab82) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:48.446118 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b4571-fece-4331-b063-a24d05dfb508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:48.447585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b4571-fece-4331-b063-a24d05dfb508 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:48.453516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 635/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:33:48 2022] GET /identity/v3/role_assignments?user.id=d115d3f3adc54a17bf80962a6ae3ab82&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=c64ac97b1795434390dc18ab162aec39 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:50.394724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8e957-a216-44cb-8ed6-afcbec48c0ab None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:50.395720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8e957-a216-44cb-8ed6-afcbec48c0ab None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:50.395720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8e957-a216-44cb-8ed6-afcbec48c0ab None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:50.397776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 626/1261] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:50.406454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967c480-4edd-4a71-a4d4-3a9607e59b26 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:50.407299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967c480-4edd-4a71-a4d4-3a9607e59b26 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:50.407898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967c480-4edd-4a71-a4d4-3a9607e59b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:50.427529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d967c480-4edd-4a71-a4d4-3a9607e59b26 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:50.428726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d967c480-4edd-4a71-a4d4-3a9607e59b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:50.502825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 636/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:50.786799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8e957-a216-44cb-8ed6-afcbec48c0ab None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:50.801546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:50.912361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8e957-a216-44cb-8ed6-afcbec48c0ab None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:50.951144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:50.951813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:50.952089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:50.952332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:50.963459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:33:50.968179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:50.969456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:50.970877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 627/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:50.978267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967c480-4edd-4a71-a4d4-3a9607e59b26 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:51.002314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:51.113297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967c480-4edd-4a71-a4d4-3a9607e59b26 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:51.154932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:51.155573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:51.155909 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:51.156386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:51.157600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:51.158026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:51.159501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:51.170828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 637/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:51.179014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:51.260253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:51.299397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00743-6b4e-4878-a506-05b0f33651db None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:51.338084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:51.338734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:51.339011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:51.339396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:51.350109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:51.351630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:51.367391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:51.368526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:51.370715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:51.370715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:51.370715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:51.370715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 628/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:33:51.376231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:51.486623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:51.509890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fe41a-31a1-4057-bb3c-b9263480f8ba None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:51.547370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:51.548079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:51.548348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:51.548623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:51.549301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:51.549608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:51.551000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:51.565625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 638/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:51.573952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:51.711847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adbf14-49e9-421c-ad00-dd3ec339c49b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:51.740559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:51.757063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:51.757704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:51.757987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:51.758266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:51.767399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:51.768884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:33:51.782494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:51.784448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:51.786802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 629/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:51.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:51.916197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89d8c3-5ef0-4146-8f8a-7a5f9de8cbe9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:51.953008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:51.953622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:51.954042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:51.954360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:51.955063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:51.955403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:51.956854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:51.967641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:51.981563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 639/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 06:33:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:33:51.989746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:52.102266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c7457-ee68-4eeb-ac6e-8eec15845e06 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:52.140544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:52.141161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:52.141561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:52.141835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:52.153573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:52.155049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:52.170837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:52.171980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:52.173279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:52.173279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:52.173279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:52.173279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 630/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:52.182585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:52.293776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d17f3f-beeb-4b47-bae8-98e24d037afc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:52.329474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:52.329718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:52.329826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:52.330695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:52.331390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:52.331702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:52.333070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:52.344480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 640/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:52.352123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:52.392234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:52.468004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903dca4b-97ea-4564-b8e3-b4a74b99e84e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:52.503155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:52.503813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:52.504097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:52.504331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:52.511344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:52.512790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:33:52.524111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:52.525166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:52.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 631/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:33:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:52.534115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:52.599366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:52.653264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e23f7-780c-4d18-b306-a2e84da3ce59 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:52.696868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f7604-b287-4fca-b7ce-0ac2c5ce98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:52.697475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7604-b287-4fca-b7ce-0ac2c5ce98c3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:52.697758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7604-b287-4fca-b7ce-0ac2c5ce98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:52.698031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7604-b287-4fca-b7ce-0ac2c5ce98c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:52.698731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f7604-b287-4fca-b7ce-0ac2c5ce98c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:52.699039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f7604-b287-4fca-b7ce-0ac2c5ce98c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:52.700538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f7604-b287-4fca-b7ce-0ac2c5ce98c3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:52.713005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 641/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:33:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:52.719622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:52.819270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:52.830058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f6e6e-f2f1-40a3-bca7-39426a881f2e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:52.866840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cf9ef-c7c1-4216-a44b-09f1d5429771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:52.867789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf9ef-c7c1-4216-a44b-09f1d5429771 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:52.868162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf9ef-c7c1-4216-a44b-09f1d5429771 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:52.868518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf9ef-c7c1-4216-a44b-09f1d5429771 None admin] PATH_INFO: `/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/d115d3f3adc54a17bf80962a6ae3ab82/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:52.872201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cf9ef-c7c1-4216-a44b-09f1d5429771 None admin] RBAC: Authorizing `identity:create_grant(project_id=f23638950c1948f2ae7a7b13c645cd54, user_id=d115d3f3adc54a17bf80962a6ae3ab82, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:52.874494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cf9ef-c7c1-4216-a44b-09f1d5429771 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:52.897621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 632/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Aug 24 06:33:52 2022] PUT /identity/v3/projects/f23638950c1948f2ae7a7b13c645cd54/users/d115d3f3adc54a17bf80962a6ae3ab82/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:33:54.765284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b8785-15b7-488c-bbd7-f07cd86a9a94 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:54.765892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b8785-15b7-488c-bbd7-f07cd86a9a94 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:54.765892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b8785-15b7-488c-bbd7-f07cd86a9a94 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:54.767305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 642/1274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:54.773587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9861c3-a284-4cdc-ad45-4d2a092fddaf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:54.774079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9861c3-a284-4cdc-ad45-4d2a092fddaf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:54.774178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9861c3-a284-4cdc-ad45-4d2a092fddaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:54.797759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6a9861c3-a284-4cdc-ad45-4d2a092fddaf None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:54.799378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6a9861c3-a284-4cdc-ad45-4d2a092fddaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:54.871003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 633/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:55.146120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b8785-15b7-488c-bbd7-f07cd86a9a94 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:55.270977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b8785-15b7-488c-bbd7-f07cd86a9a94 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:55.288787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:55.317936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:55.318913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:55.319412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:55.319866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:55.334445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:33:55.339131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:55.340224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:55.344262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 643/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Aug 24 06:33:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:55.346890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9861c3-a284-4cdc-ad45-4d2a092fddaf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:55.429656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:55.452781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9861c3-a284-4cdc-ad45-4d2a092fddaf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:55.489545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:55.490135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:55.490411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:55.490678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:55.491373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:55.491674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:55.493046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:55.502600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 634/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Aug 24 06:33:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:55.508624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:55.615745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839022c9-48d3-4837-b92f-5a696238af1d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:55.658234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:55.658872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:55.659155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:55.659457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:55.674645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:55.677060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:55.698552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:55.699758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:55.700914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:55.700914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:55.700914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:55.700914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 644/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:55.708295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:55.821347 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee518ca-13c5-466e-a47d-40d2a36f35fe None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:55.859173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:55.859822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:55.860295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:55.860572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:55.861213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:55.861480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:55.862841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:55.874591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 635/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:55.883906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:55.918553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:55.990869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8885d0d-9a55-4171-a2fc-a9c5d15a50a6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:56.026900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:56.027564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:56.027851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:56.028128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:56.035590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:56.037213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:33:56.050951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:56.052144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:56.053978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 645/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:56.061602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:56.068263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:56.193741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd80977-e6fe-4c93-ab56-851209b44ebb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:56.236366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:56.236988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:56.237274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:56.237550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:56.238283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:56.238590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:56.240068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:56.257426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 636/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Aug 24 06:33:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:56.263444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:56.374120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:56.390915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e856c92-5600-4c8d-b530-3e6cd4aba7fd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:56.431556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:56.432161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:56.432445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:56.432722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:56.504605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:56.506066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:56.614735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:56.625874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 646/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Aug 24 06:33:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:33:56.632590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:56.704055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:56.744014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e718652-4812-41ec-89be-02ce00d298d5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:56.780948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:56.781569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:56.781922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:56.782251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:56.792564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:56.794015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:33:56.809013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:56.810117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:33:56.811553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:33:56.811553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:33:56.811553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:56.811553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 637/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:56.819958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:56.928685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c3acd-bf37-4060-bfe1-f188f739cf63 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:56.964937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:56.965539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:56.965822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:56.966099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:56.966749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:56.967055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:56.968472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:56.980100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 647/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Aug 24 06:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:56.986640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:57.114331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8f8b6-3fd2-4519-a937-7bf323f99f71 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:57.128187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:57.150484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:57.151105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:57.151422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:57.151703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:57.158781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:57.160269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:33:57.173212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:33:57.174263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 24 06:33:57.176021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 638/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Aug 24 06:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:33:57.184052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:57.240054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:57.295060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288577e-37fd-4ae0-870a-cfce5a697e88 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:57.329760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a12ee1-c119-4846-a4d9-bf971ac2747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:57.330374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12ee1-c119-4846-a4d9-bf971ac2747e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:57.330660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12ee1-c119-4846-a4d9-bf971ac2747e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:57.330938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a12ee1-c119-4846-a4d9-bf971ac2747e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:57.331677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a12ee1-c119-4846-a4d9-bf971ac2747e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:57.331995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a12ee1-c119-4846-a4d9-bf971ac2747e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:57.333449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a12ee1-c119-4846-a4d9-bf971ac2747e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:57.346291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 648/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Aug 24 06:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:57.353370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:57.461223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f9f9f-1ab6-4d7f-8de9-93bbebf3a535 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:57.496636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec77fc-5e2f-4f3c-88dd-95b6b65a3fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:57.497221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec77fc-5e2f-4f3c-88dd-95b6b65a3fea None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:57.497502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec77fc-5e2f-4f3c-88dd-95b6b65a3fea None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:57.497775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec77fc-5e2f-4f3c-88dd-95b6b65a3fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:57.498503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec77fc-5e2f-4f3c-88dd-95b6b65a3fea None admin] RBAC: Adding query filter params (role.id=c64ac97b1795434390dc18ab162aec39, scope.project.id=f23638950c1948f2ae7a7b13c645cd54, user.id=d115d3f3adc54a17bf80962a6ae3ab82) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:33:57.498810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec77fc-5e2f-4f3c-88dd-95b6b65a3fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:57.500277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec77fc-5e2f-4f3c-88dd-95b6b65a3fea None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:57.506455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 639/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Aug 24 06:33:57 2022] GET /identity/v3/role_assignments?user.id=d115d3f3adc54a17bf80962a6ae3ab82&scope.project.id=f23638950c1948f2ae7a7b13c645cd54&role.id=c64ac97b1795434390dc18ab162aec39 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:33:59.431999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0504a-070c-4dd4-a904-c9e4023ae6d4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:59.432565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0504a-070c-4dd4-a904-c9e4023ae6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:59.432565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0504a-070c-4dd4-a904-c9e4023ae6d4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:59.433906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 649/1288] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:59.440721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed1fe8-f3de-4b97-9360-b33dfff5ffd0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:59.441447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed1fe8-f3de-4b97-9360-b33dfff5ffd0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:59.441601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed1fe8-f3de-4b97-9360-b33dfff5ffd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:59.472050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-91ed1fe8-f3de-4b97-9360-b33dfff5ffd0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:59.473179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-91ed1fe8-f3de-4b97-9360-b33dfff5ffd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:59.507396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:59.550488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 640/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:33:59.841568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0504a-070c-4dd4-a904-c9e4023ae6d4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:59.949678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0504a-070c-4dd4-a904-c9e4023ae6d4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:59.960557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:59.987173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:59.987848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:59.988135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:59.988488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:59.989013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:59.990423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:33:59.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:34:00.001180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:34:00.002261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:34:00.006220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 650/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:33:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:34:00.007942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed1fe8-f3de-4b97-9360-b33dfff5ffd0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:00.120846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed1fe8-f3de-4b97-9360-b33dfff5ffd0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:00.156495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2dbfe-84da-4544-affd-1e88b2a802ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:00.157143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2dbfe-84da-4544-affd-1e88b2a802ee None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:00.157424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2dbfe-84da-4544-affd-1e88b2a802ee None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:00.157700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2dbfe-84da-4544-affd-1e88b2a802ee None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:00.158339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2dbfe-84da-4544-affd-1e88b2a802ee None admin] RBAC: Adding query filter params (name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:34:00.158648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2dbfe-84da-4544-affd-1e88b2a802ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:00.160057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2dbfe-84da-4544-affd-1e88b2a802ee None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:00.165008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 641/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 06:34:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:34:00.174632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:00.289138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbd6d2-8ca0-4f1e-8097-500e9b425d65 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:00.327886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a57140-ffcc-4187-bef1-30f17d4b888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:00.328994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57140-ffcc-4187-bef1-30f17d4b888d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:00.329304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57140-ffcc-4187-bef1-30f17d4b888d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:00.329585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57140-ffcc-4187-bef1-30f17d4b888d None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:00.330231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a57140-ffcc-4187-bef1-30f17d4b888d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:34:00.330545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a57140-ffcc-4187-bef1-30f17d4b888d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:00.331983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a57140-ffcc-4187-bef1-30f17d4b888d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:00.337078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 651/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 06:34:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:34:02.225013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff965b-096c-4db7-9baf-3bbeaab72ec0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:02.225512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff965b-096c-4db7-9baf-3bbeaab72ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:02.225601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff965b-096c-4db7-9baf-3bbeaab72ec0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:02.227781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 642/1293] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:34:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:02.233428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69560cff-bc8b-460b-8fdf-60e2bb890d62 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:02.233893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69560cff-bc8b-460b-8fdf-60e2bb890d62 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:02.234005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69560cff-bc8b-460b-8fdf-60e2bb890d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:02.254675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-69560cff-bc8b-460b-8fdf-60e2bb890d62 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:02.255908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-69560cff-bc8b-460b-8fdf-60e2bb890d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:02.293729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:02.335119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 652/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:34:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:34:02.605489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff965b-096c-4db7-9baf-3bbeaab72ec0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:02.642752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:02.730698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff965b-096c-4db7-9baf-3bbeaab72ec0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:02.766458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c88707-721e-4f6b-ad04-4453231be4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:02.767407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c88707-721e-4f6b-ad04-4453231be4fe None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:02.767687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c88707-721e-4f6b-ad04-4453231be4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:02.767959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c88707-721e-4f6b-ad04-4453231be4fe None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:02.768615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c88707-721e-4f6b-ad04-4453231be4fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:02.769991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c88707-721e-4f6b-ad04-4453231be4fe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:02.787208 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 643/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Aug 24 06:34:02 2022] POST /identity/v3/services => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:34:04.689411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e6de3-2f23-47f7-aedc-f38446322934 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:04.690010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e6de3-2f23-47f7-aedc-f38446322934 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:04.690010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e6de3-2f23-47f7-aedc-f38446322934 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:04.691376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 653/1296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:04.698155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d9352-7a92-418e-aaf4-d9e5e5a99d70 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:04.698915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d9352-7a92-418e-aaf4-d9e5e5a99d70 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:04.699040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d9352-7a92-418e-aaf4-d9e5e5a99d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:04.726250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-be3d9352-7a92-418e-aaf4-d9e5e5a99d70 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:04.727526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-be3d9352-7a92-418e-aaf4-d9e5e5a99d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:04.814278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 644/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:34:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:34:05.081316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e6de3-2f23-47f7-aedc-f38446322934 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:05.190133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e6de3-2f23-47f7-aedc-f38446322934 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:05.226930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:05.227702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:05.228013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:05.228302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:05.228869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:05.230289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:34:05.240295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:34:05.241450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:34:05.242424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 654/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:34:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:34:05.248446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d9352-7a92-418e-aaf4-d9e5e5a99d70 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:05.268051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:05.351459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d9352-7a92-418e-aaf4-d9e5e5a99d70 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:05.387446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:05.388048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:05.388322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:05.388592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:05.389220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:34:05.389522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:05.390887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:05.396077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 645/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 06:34:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:34:05.402611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:05.442207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:05.514206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddb884-eec1-45bb-9231-2f6f3fccfaf3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:05.549858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066872d-01ce-4f52-8a0a-351b6f568070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:05.550462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066872d-01ce-4f52-8a0a-351b6f568070 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:05.550747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066872d-01ce-4f52-8a0a-351b6f568070 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:05.551026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066872d-01ce-4f52-8a0a-351b6f568070 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:05.551769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066872d-01ce-4f52-8a0a-351b6f568070 None admin] RBAC: Adding query filter params (interface=public, service_id=fcb7951e02f748018505c59c4ec24aed, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:34:05.552086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066872d-01ce-4f52-8a0a-351b6f568070 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:05.553471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066872d-01ce-4f52-8a0a-351b6f568070 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:05.559225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 655/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Aug 24 06:34:05 2022] GET /identity/v3/endpoints?service_id=fcb7951e02f748018505c59c4ec24aed&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:34:05.566050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:05.652080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:05.671838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582946b0-310c-4365-a8d2-e5ee7534b9d9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:05.717758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861f338-d6fa-48f6-8fb7-78cca0aa2dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:05.718599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f338-d6fa-48f6-8fb7-78cca0aa2dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:05.718728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f338-d6fa-48f6-8fb7-78cca0aa2dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:05.718893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f338-d6fa-48f6-8fb7-78cca0aa2dc1 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:05.719620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861f338-d6fa-48f6-8fb7-78cca0aa2dc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:05.721011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861f338-d6fa-48f6-8fb7-78cca0aa2dc1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:05.728036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 646/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Aug 24 06:34:05 2022] GET /identity/v3/services => generated 2191 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:34:07.608100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfc96b-b6c8-4301-974f-7e8a87eabefa None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:07.608716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfc96b-b6c8-4301-974f-7e8a87eabefa None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:07.608716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfc96b-b6c8-4301-974f-7e8a87eabefa None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:07.610072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 656/1302] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:07.616529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a475-ed64-4879-8de3-af454a594369 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:07.617252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a475-ed64-4879-8de3-af454a594369 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:07.617404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a475-ed64-4879-8de3-af454a594369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:07.641615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d428a475-ed64-4879-8de3-af454a594369 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:07.642914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d428a475-ed64-4879-8de3-af454a594369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:07.716154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 647/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:34:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:34:07.989859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfc96b-b6c8-4301-974f-7e8a87eabefa None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:08.099846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:08.111521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfc96b-b6c8-4301-974f-7e8a87eabefa None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:08.147381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:08.148013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:08.148296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:08.148567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] PATH_INFO: `/v3/services/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:08.149157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:08.150542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:34:08.159614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:34:08.160852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Aug 24 06:34:08.162098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 657/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Aug 24 06:34:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:34:08.169124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428a475-ed64-4879-8de3-af454a594369 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:08.260040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:08.287171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428a475-ed64-4879-8de3-af454a594369 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:08.324757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d4aba-bdd7-4b60-bd1c-e215c6e08afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:08.325360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d4aba-bdd7-4b60-bd1c-e215c6e08afb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:08.325636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d4aba-bdd7-4b60-bd1c-e215c6e08afb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:08.325911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d4aba-bdd7-4b60-bd1c-e215c6e08afb None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:08.326549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d4aba-bdd7-4b60-bd1c-e215c6e08afb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:34:08.326859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d4aba-bdd7-4b60-bd1c-e215c6e08afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:08.328258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d4aba-bdd7-4b60-bd1c-e215c6e08afb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:08.334233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 648/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Aug 24 06:34:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:34:08.340960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:08.462641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea2581-1e6f-4d65-9ed2-4f1f557ef8ee None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:08.499524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257773fa-6dac-43e3-9f4f-3a28e6278ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:08.500130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257773fa-6dac-43e3-9f4f-3a28e6278ca8 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:08.500503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257773fa-6dac-43e3-9f4f-3a28e6278ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:08.501015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257773fa-6dac-43e3-9f4f-3a28e6278ca8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:08.501713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257773fa-6dac-43e3-9f4f-3a28e6278ca8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:08.503095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257773fa-6dac-43e3-9f4f-3a28e6278ca8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:08.530906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 658/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:34:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:34:18.331178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=30 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:18.526133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:13.663805 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ae05e-2c5f-4a22-b21c-780d298c3bb9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:13.665303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ae05e-2c5f-4a22-b21c-780d298c3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:13.666207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ae05e-2c5f-4a22-b21c-780d298c3bb9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:13.675321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4a361-8d52-492c-be81-7008f8cfd84a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:13.675770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4a361-8d52-492c-be81-7008f8cfd84a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:13.675879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4a361-8d52-492c-be81-7008f8cfd84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:13.680479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 649/1307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:13.821995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f7e4a361-8d52-492c-be81-7008f8cfd84a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:13.823802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f7e4a361-8d52-492c-be81-7008f8cfd84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:13.975229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 659/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:35:14.260879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3b66-56cf-49a0-b8d8-ea73a3de7a1c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:14.261239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3b66-56cf-49a0-b8d8-ea73a3de7a1c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:14.261738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3b66-56cf-49a0-b8d8-ea73a3de7a1c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:14.263252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 650/1309] 10.222.0.52 () {54 vars in 965 bytes} [Wed Aug 24 06:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:14.269888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bec746-3f18-46ed-b8af-db18c156560d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:14.270211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bec746-3f18-46ed-b8af-db18c156560d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:14.270470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bec746-3f18-46ed-b8af-db18c156560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:14.360584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-30bec746-3f18-46ed-b8af-db18c156560d None None] MFA Rules not processed for user `2fc4592ef54b41059fa7767a5ca3aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:14.361949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-30bec746-3f18-46ed-b8af-db18c156560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:14.385777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:14.451441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 660/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Wed Aug 24 06:35:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:35:14.459569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e3b66-56cf-49a0-b8d8-ea73a3de7a1c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:14.461332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f14e3b66-56cf-49a0-b8d8-ea73a3de7a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:14.587120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e3b66-56cf-49a0-b8d8-ea73a3de7a1c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:14.667081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cca526-fe01-4431-aa0a-9d00dc871326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:14.667780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cca526-fe01-4431-aa0a-9d00dc871326 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:14.667999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cca526-fe01-4431-aa0a-9d00dc871326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:14.668476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cca526-fe01-4431-aa0a-9d00dc871326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:14.670531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-42cca526-fe01-4431-aa0a-9d00dc871326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:14.731887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:14.796249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cca526-fe01-4431-aa0a-9d00dc871326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:14.797714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cca526-fe01-4431-aa0a-9d00dc871326 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:14.817681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 651/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:17.012950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e993-c9de-42eb-803a-d89dceffa4fc None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.013885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e993-c9de-42eb-803a-d89dceffa4fc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.014166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38e993-c9de-42eb-803a-d89dceffa4fc None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.015591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 661/1312] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:17.021505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949c3c6-2287-48b1-9740-adc0667c4763 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.021963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949c3c6-2287-48b1-9740-adc0667c4763 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.022267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949c3c6-2287-48b1-9740-adc0667c4763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.112330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7949c3c6-2287-48b1-9740-adc0667c4763 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:17.113747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7949c3c6-2287-48b1-9740-adc0667c4763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:17.220662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 652/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:17.523111 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11561-dd9e-43fa-9387-78aad5f6bdbb None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.523398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11561-dd9e-43fa-9387-78aad5f6bdbb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.523713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a11561-dd9e-43fa-9387-78aad5f6bdbb None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.525126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 662/1314] 10.222.0.52 () {54 vars in 965 bytes} [Wed Aug 24 06:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:17.531313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4cb98-9ef5-403e-84c3-5582ba7fe424 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.531610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4cb98-9ef5-403e-84c3-5582ba7fe424 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.531895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4cb98-9ef5-403e-84c3-5582ba7fe424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.544179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:17.613615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7ed4cb98-9ef5-403e-84c3-5582ba7fe424 None None] MFA Rules not processed for user `2fc4592ef54b41059fa7767a5ca3aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:17.614877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7ed4cb98-9ef5-403e-84c3-5582ba7fe424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:17.702434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 653/1315] 10.222.0.52 () {58 vars in 1062 bytes} [Wed Aug 24 06:35:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:17.712020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a11561-dd9e-43fa-9387-78aad5f6bdbb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:17.713617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e9a11561-dd9e-43fa-9387-78aad5f6bdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:17.823114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a11561-dd9e-43fa-9387-78aad5f6bdbb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:17.904939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:17.907627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb67b4-0f28-405f-a4a2-a0713616b8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:17.908381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb67b4-0f28-405f-a4a2-a0713616b8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.908640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb67b4-0f28-405f-a4a2-a0713616b8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.908994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb67b4-0f28-405f-a4a2-a0713616b8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.911394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-02eb67b4-0f28-405f-a4a2-a0713616b8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:18.038376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb67b4-0f28-405f-a4a2-a0713616b8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:18.039939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb67b4-0f28-405f-a4a2-a0713616b8d3 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:18.062156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 663/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:20.515945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fabbb2-5e9c-4f90-8de8-46f66fd33760 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:20.516515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fabbb2-5e9c-4f90-8de8-46f66fd33760 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:20.516515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fabbb2-5e9c-4f90-8de8-46f66fd33760 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:20.518036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 654/1317] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:20.526749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68940bd9-66ab-4660-afda-0f060ba476ec None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:20.527240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68940bd9-66ab-4660-afda-0f060ba476ec None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:20.527362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68940bd9-66ab-4660-afda-0f060ba476ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:20.605707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-68940bd9-66ab-4660-afda-0f060ba476ec None None] MFA Rules not processed for user `636cfdf20b5042e8a116abd580b78f06`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:20.606935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-68940bd9-66ab-4660-afda-0f060ba476ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:20.678618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:20.726255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 664/1318] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:21.017063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fabbb2-5e9c-4f90-8de8-46f66fd33760 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:21.018701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a1fabbb2-5e9c-4f90-8de8-46f66fd33760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:21.127730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fabbb2-5e9c-4f90-8de8-46f66fd33760 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:21.130546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:21.204092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262dcce4-2a18-4730-ab2a-75ed1c4f9a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:21.204708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dcce4-2a18-4730-ab2a-75ed1c4f9a6d service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:21.205054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dcce4-2a18-4730-ab2a-75ed1c4f9a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:21.205327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dcce4-2a18-4730-ab2a-75ed1c4f9a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:21.206923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-262dcce4-2a18-4730-ab2a-75ed1c4f9a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:21.335145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262dcce4-2a18-4730-ab2a-75ed1c4f9a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:21.336561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262dcce4-2a18-4730-ab2a-75ed1c4f9a6d service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:21.354841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 655/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:23.938418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9532474-66c9-4cd6-afcb-5fc1636e21c3 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:23.939157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9532474-66c9-4cd6-afcb-5fc1636e21c3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:23.939157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9532474-66c9-4cd6-afcb-5fc1636e21c3 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:23.940504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 665/1320] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:23.947445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fe3ae-3d21-4583-b7cd-84635af8b83c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:23.947919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fe3ae-3d21-4583-b7cd-84635af8b83c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:23.948028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fe3ae-3d21-4583-b7cd-84635af8b83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:23.999634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:24.026102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ed5fe3ae-3d21-4583-b7cd-84635af8b83c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:24.027358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ed5fe3ae-3d21-4583-b7cd-84635af8b83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:24.131997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 656/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:24.415225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9532474-66c9-4cd6-afcb-5fc1636e21c3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:24.416820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e9532474-66c9-4cd6-afcb-5fc1636e21c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:24.528386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9532474-66c9-4cd6-afcb-5fc1636e21c3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:24.616677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdeea6f-023a-47ba-9630-25ba12e93dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:24.617295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeea6f-023a-47ba-9630-25ba12e93dad service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:24.617558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeea6f-023a-47ba-9630-25ba12e93dad service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:24.617848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdeea6f-023a-47ba-9630-25ba12e93dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:24.619508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2fdeea6f-023a-47ba-9630-25ba12e93dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:24.627818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:24.739554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdeea6f-023a-47ba-9630-25ba12e93dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:24.741007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdeea6f-023a-47ba-9630-25ba12e93dad service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:24.759806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 666/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:27.365002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6ac2-ba05-4160-9285-5005ea5c5393 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:27.365551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6ac2-ba05-4160-9285-5005ea5c5393 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:27.365551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e6ac2-ba05-4160-9285-5005ea5c5393 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:27.366850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 657/1323] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:27.372382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701b75c-35af-44fc-9446-0d3ed0674014 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:27.372825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701b75c-35af-44fc-9446-0d3ed0674014 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:27.372930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701b75c-35af-44fc-9446-0d3ed0674014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:27.446725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0701b75c-35af-44fc-9446-0d3ed0674014 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:27.448007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0701b75c-35af-44fc-9446-0d3ed0674014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:27.509461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:27.551205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 667/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:27.840677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e6ac2-ba05-4160-9285-5005ea5c5393 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:27.843150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7d2e6ac2-ba05-4160-9285-5005ea5c5393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:27.957245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e6ac2-ba05-4160-9285-5005ea5c5393 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:27.970087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:28.038506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4fce8-5cbf-472d-8f2d-e1d8349679f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:28.039119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4fce8-5cbf-472d-8f2d-e1d8349679f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:28.039436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4fce8-5cbf-472d-8f2d-e1d8349679f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:28.039709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4fce8-5cbf-472d-8f2d-e1d8349679f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:28.041322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9da4fce8-5cbf-472d-8f2d-e1d8349679f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:28.156083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4fce8-5cbf-472d-8f2d-e1d8349679f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:28.157385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4fce8-5cbf-472d-8f2d-e1d8349679f7 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:28.176990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 658/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:30.167743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e8aee-77d9-475a-b7a7-ac63805d63c3 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:30.168286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e8aee-77d9-475a-b7a7-ac63805d63c3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:30.168286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e8aee-77d9-475a-b7a7-ac63805d63c3 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:30.169528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 668/1326] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:30.175568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fadc56-ffd6-4069-b778-6f8799a8db27 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:30.176083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fadc56-ffd6-4069-b778-6f8799a8db27 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:30.176179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fadc56-ffd6-4069-b778-6f8799a8db27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:30.230139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:30.249543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-67fadc56-ffd6-4069-b778-6f8799a8db27 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:30.250770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-67fadc56-ffd6-4069-b778-6f8799a8db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:30.350568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 659/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:30.632096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e8aee-77d9-475a-b7a7-ac63805d63c3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:30.633656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-794e8aee-77d9-475a-b7a7-ac63805d63c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:30.757768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e8aee-77d9-475a-b7a7-ac63805d63c3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:30.833896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319bec0-9764-4f84-8f85-c93b1f685910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:30.834507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319bec0-9764-4f84-8f85-c93b1f685910 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:30.834788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319bec0-9764-4f84-8f85-c93b1f685910 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:30.835061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319bec0-9764-4f84-8f85-c93b1f685910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:30.836687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f319bec0-9764-4f84-8f85-c93b1f685910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:30.854701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:30.954130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319bec0-9764-4f84-8f85-c93b1f685910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:30.955722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319bec0-9764-4f84-8f85-c93b1f685910 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:30.974547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 669/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:32.962521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd9f0-8570-4d5a-a0a6-2947527e8a93 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:32.963011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd9f0-8570-4d5a-a0a6-2947527e8a93 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:32.963121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd9f0-8570-4d5a-a0a6-2947527e8a93 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:32.964476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 660/1329] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:32.970120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8934620-2fdb-432d-8e35-a90e34003632 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:32.970851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8934620-2fdb-432d-8e35-a90e34003632 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:32.970851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8934620-2fdb-432d-8e35-a90e34003632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:33.046370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c8934620-2fdb-432d-8e35-a90e34003632 None None] MFA Rules not processed for user `636cfdf20b5042e8a116abd580b78f06`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:33.047631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c8934620-2fdb-432d-8e35-a90e34003632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:33.115781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:33.151924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 670/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:33.437232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadd9f0-8570-4d5a-a0a6-2947527e8a93 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:33.438653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2aadd9f0-8570-4d5a-a0a6-2947527e8a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:33.557297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadd9f0-8570-4d5a-a0a6-2947527e8a93 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:33.577519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:33.637257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4895a30-79fa-4954-b186-f50f26bc9730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:33.637887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4895a30-79fa-4954-b186-f50f26bc9730 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:33.638165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4895a30-79fa-4954-b186-f50f26bc9730 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:33.638423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4895a30-79fa-4954-b186-f50f26bc9730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:33.640120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d4895a30-79fa-4954-b186-f50f26bc9730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:33.762226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4895a30-79fa-4954-b186-f50f26bc9730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:33.763868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4895a30-79fa-4954-b186-f50f26bc9730 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:33.784403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 661/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:35.995442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419cbbb-88b5-4dd7-a729-5a8cdccb3da7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:35.996328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419cbbb-88b5-4dd7-a729-5a8cdccb3da7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:35.996328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419cbbb-88b5-4dd7-a729-5a8cdccb3da7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:35.998474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 671/1332] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:36.005022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9514690-57b9-4268-961d-27517616f910 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:36.005714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9514690-57b9-4268-961d-27517616f910 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:36.005861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9514690-57b9-4268-961d-27517616f910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:36.080406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:36.083434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e9514690-57b9-4268-961d-27517616f910 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:36.085673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e9514690-57b9-4268-961d-27517616f910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:36.182272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 662/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:36.470545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419cbbb-88b5-4dd7-a729-5a8cdccb3da7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:36.473357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c419cbbb-88b5-4dd7-a729-5a8cdccb3da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:36.587396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419cbbb-88b5-4dd7-a729-5a8cdccb3da7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:36.665192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffa851-05c0-454e-8d31-1ad82cd95634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:36.665810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffa851-05c0-454e-8d31-1ad82cd95634 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:36.666089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffa851-05c0-454e-8d31-1ad82cd95634 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:36.666362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffa851-05c0-454e-8d31-1ad82cd95634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:36.667995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-38ffa851-05c0-454e-8d31-1ad82cd95634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:36.698834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:36.783720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffa851-05c0-454e-8d31-1ad82cd95634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:36.785187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffa851-05c0-454e-8d31-1ad82cd95634 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:36.807218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 672/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:38.779561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9529c40-5a96-4fcb-b8ff-c3f19d12868c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:38.780216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9529c40-5a96-4fcb-b8ff-c3f19d12868c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:38.780216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9529c40-5a96-4fcb-b8ff-c3f19d12868c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:38.781496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 663/1335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:38.788491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c642a-0f0e-47ce-805a-5138a907b900 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:38.789248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c642a-0f0e-47ce-805a-5138a907b900 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:38.789436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c642a-0f0e-47ce-805a-5138a907b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:38.878231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1e6c642a-0f0e-47ce-805a-5138a907b900 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:38.879515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1e6c642a-0f0e-47ce-805a-5138a907b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:38.961180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:38.989326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 673/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:39.277915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9529c40-5a96-4fcb-b8ff-c3f19d12868c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:39.279517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d9529c40-5a96-4fcb-b8ff-c3f19d12868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:39.389760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9529c40-5a96-4fcb-b8ff-c3f19d12868c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:39.419085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:39.476425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca1578-56b0-473a-a155-93e115cf876d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:39.477035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca1578-56b0-473a-a155-93e115cf876d service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:39.477316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca1578-56b0-473a-a155-93e115cf876d service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:39.477591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca1578-56b0-473a-a155-93e115cf876d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:39.479212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e3ca1578-56b0-473a-a155-93e115cf876d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:39.611919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca1578-56b0-473a-a155-93e115cf876d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:39.613384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca1578-56b0-473a-a155-93e115cf876d service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:39.632785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 664/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:41.890706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae29e70-1b96-4dd6-a62b-823c5108ecb4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:41.891675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae29e70-1b96-4dd6-a62b-823c5108ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:41.891675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae29e70-1b96-4dd6-a62b-823c5108ecb4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:41.893173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 674/1338] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:41.900735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cceaa-c591-4a3a-b798-57f6cd0d7984 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:41.901246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cceaa-c591-4a3a-b798-57f6cd0d7984 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:41.901356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cceaa-c591-4a3a-b798-57f6cd0d7984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:41.976813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-914cceaa-c591-4a3a-b798-57f6cd0d7984 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:41.978056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-914cceaa-c591-4a3a-b798-57f6cd0d7984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:41.986664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:42.086506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 665/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:42.382107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae29e70-1b96-4dd6-a62b-823c5108ecb4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:42.384795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cae29e70-1b96-4dd6-a62b-823c5108ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:42.501736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae29e70-1b96-4dd6-a62b-823c5108ecb4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:42.589761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641cc81-5538-428b-bfca-e9e83f87e392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:42.590372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641cc81-5538-428b-bfca-e9e83f87e392 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:42.590656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641cc81-5538-428b-bfca-e9e83f87e392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:42.590934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641cc81-5538-428b-bfca-e9e83f87e392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:42.592587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7641cc81-5538-428b-bfca-e9e83f87e392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:42.637121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:42.720398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641cc81-5538-428b-bfca-e9e83f87e392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:42.722438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641cc81-5538-428b-bfca-e9e83f87e392 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:42.742429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 675/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:46.678479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea49449-7170-45a0-a951-e96327d829e1 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:46.679070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea49449-7170-45a0-a951-e96327d829e1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:46.679070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea49449-7170-45a0-a951-e96327d829e1 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:46.680431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 666/1341] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:46.685878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab91e9-8b54-484e-a2d0-2bf46a5d259a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:46.686339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab91e9-8b54-484e-a2d0-2bf46a5d259a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:46.686456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab91e9-8b54-484e-a2d0-2bf46a5d259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:46.754439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-82ab91e9-8b54-484e-a2d0-2bf46a5d259a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:46.755686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-82ab91e9-8b54-484e-a2d0-2bf46a5d259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:46.846547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:46.859699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 676/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:47.137954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea49449-7170-45a0-a951-e96327d829e1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:47.139610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ea49449-7170-45a0-a951-e96327d829e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:47.251648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea49449-7170-45a0-a951-e96327d829e1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:47.292486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:47.331717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c795ed2-2b5f-4d9d-85ae-b59148939a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:47.332334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795ed2-2b5f-4d9d-85ae-b59148939a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:47.332619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795ed2-2b5f-4d9d-85ae-b59148939a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:47.332903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c795ed2-2b5f-4d9d-85ae-b59148939a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:47.334504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0c795ed2-2b5f-4d9d-85ae-b59148939a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:47.445657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c795ed2-2b5f-4d9d-85ae-b59148939a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:47.447028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c795ed2-2b5f-4d9d-85ae-b59148939a48 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:47.465505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 667/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:50.051313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec2b98-4183-4d3a-a3d9-ce86e81e80d2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:50.051920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec2b98-4183-4d3a-a3d9-ce86e81e80d2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:50.051920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec2b98-4183-4d3a-a3d9-ce86e81e80d2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:50.053245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 677/1344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:50.058550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327103b7-8b53-4564-a9ca-7a0089062497 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:50.059017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327103b7-8b53-4564-a9ca-7a0089062497 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:50.059168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327103b7-8b53-4564-a9ca-7a0089062497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:50.129901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-327103b7-8b53-4564-a9ca-7a0089062497 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:50.131099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-327103b7-8b53-4564-a9ca-7a0089062497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:50.151093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:50.235785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 668/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:50.512974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec2b98-4183-4d3a-a3d9-ce86e81e80d2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:50.514557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-37ec2b98-4183-4d3a-a3d9-ce86e81e80d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:50.623225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec2b98-4183-4d3a-a3d9-ce86e81e80d2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:50.711169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22a759-acb9-4895-aa6e-61ffd1a81387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:50.711809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22a759-acb9-4895-aa6e-61ffd1a81387 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:50.712087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22a759-acb9-4895-aa6e-61ffd1a81387 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:50.712645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22a759-acb9-4895-aa6e-61ffd1a81387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:50.714384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7b22a759-acb9-4895-aa6e-61ffd1a81387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:50.759917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:50.827826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22a759-acb9-4895-aa6e-61ffd1a81387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:50.829278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22a759-acb9-4895-aa6e-61ffd1a81387 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:50.848365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 678/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:54.914613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbdeab-ef5d-4eea-8703-8e2f9a746f92 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:54.915258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbdeab-ef5d-4eea-8703-8e2f9a746f92 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:54.915584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbdeab-ef5d-4eea-8703-8e2f9a746f92 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:54.923247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 669/1347] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:35:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:54.927164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bf72e-8680-45c3-a0ca-f0f63c08d071 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:54.927700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bf72e-8680-45c3-a0ca-f0f63c08d071 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:54.927799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bf72e-8680-45c3-a0ca-f0f63c08d071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:55.019145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-163bf72e-8680-45c3-a0ca-f0f63c08d071 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:55.020432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-163bf72e-8680-45c3-a0ca-f0f63c08d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:55.126881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:55.130068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 679/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:55.412816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbdeab-ef5d-4eea-8703-8e2f9a746f92 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:55.414459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-47bbdeab-ef5d-4eea-8703-8e2f9a746f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:55.535851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbdeab-ef5d-4eea-8703-8e2f9a746f92 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:55.589904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:55.620572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52444ae3-2403-467b-9273-b096065acbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:55.621195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444ae3-2403-467b-9273-b096065acbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:55.621480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444ae3-2403-467b-9273-b096065acbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:55.621757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444ae3-2403-467b-9273-b096065acbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:55.623408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-52444ae3-2403-467b-9273-b096065acbd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:55.743852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52444ae3-2403-467b-9273-b096065acbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:55.745332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52444ae3-2403-467b-9273-b096065acbd1 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:55.767003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 670/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:00.759879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f814508-dd1c-40d9-8f88-358ad86d3050 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:00.760311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f814508-dd1c-40d9-8f88-358ad86d3050 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:00.760589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f814508-dd1c-40d9-8f88-358ad86d3050 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:00.762382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 680/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:00.768292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0dd80-f219-41b1-b1f3-1f98baf60754 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:00.768796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0dd80-f219-41b1-b1f3-1f98baf60754 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:00.769186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0dd80-f219-41b1-b1f3-1f98baf60754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:00.858605 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-00c0dd80-f219-41b1-b1f3-1f98baf60754 None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:00.860161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-00c0dd80-f219-41b1-b1f3-1f98baf60754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:00.890135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:00.999025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 671/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:36:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:05.099243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5ec4e-685b-47e8-b5ca-ddde7fdd3fe3 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:05.099771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5ec4e-685b-47e8-b5ca-ddde7fdd3fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:05.099852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5ec4e-685b-47e8-b5ca-ddde7fdd3fe3 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:05.101220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 681/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:05.106346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916943-bc9b-417b-913e-9df282b7ac52 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:05.106763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916943-bc9b-417b-913e-9df282b7ac52 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:05.106956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916943-bc9b-417b-913e-9df282b7ac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:05.186840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5e916943-bc9b-417b-913e-9df282b7ac52 None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:05.188096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5e916943-bc9b-417b-913e-9df282b7ac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:05.269480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:05.309521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 672/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:36:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:06.827673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb15f2-63f0-4127-bfe2-06f880fffd99 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:06.828819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb15f2-63f0-4127-bfe2-06f880fffd99 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:06.828819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb15f2-63f0-4127-bfe2-06f880fffd99 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:06.829609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 682/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:06.835147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56ea2c-b54a-41c6-b311-9c56e9b07478 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:06.835672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56ea2c-b54a-41c6-b311-9c56e9b07478 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:06.835755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56ea2c-b54a-41c6-b311-9c56e9b07478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:06.913169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3a56ea2c-b54a-41c6-b311-9c56e9b07478 None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:06.914383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3a56ea2c-b54a-41c6-b311-9c56e9b07478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:07.022223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 673/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:36:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:10.387483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16ede3-af00-4f02-ab53-ec0fbf6f2202 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:10.388060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16ede3-af00-4f02-ab53-ec0fbf6f2202 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:10.388060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16ede3-af00-4f02-ab53-ec0fbf6f2202 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:10.389390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 683/1356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:10.394896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5654-b766-428f-ab7c-0732edcebdfe None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:10.395370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5654-b766-428f-ab7c-0732edcebdfe None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:10.395463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5654-b766-428f-ab7c-0732edcebdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:10.434329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:10.490352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-74ba5654-b766-428f-ab7c-0732edcebdfe None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:10.491628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-74ba5654-b766-428f-ab7c-0732edcebdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:10.630076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 674/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:10.946331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d1e5-0641-4bb9-9da6-02f88938099e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:10.946618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d1e5-0641-4bb9-9da6-02f88938099e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:10.947077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84d1e5-0641-4bb9-9da6-02f88938099e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:10.948463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 684/1358] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 06:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:10.958826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e141b7-722a-47fe-ae3e-d023f32686da None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:10.959269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e141b7-722a-47fe-ae3e-d023f32686da None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:10.959647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e141b7-722a-47fe-ae3e-d023f32686da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:11.040177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f2e141b7-722a-47fe-ae3e-d023f32686da None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:11.041672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f2e141b7-722a-47fe-ae3e-d023f32686da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:11.094034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:11.165188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 675/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 06:36:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:11.172942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d1e5-0641-4bb9-9da6-02f88938099e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:11.174492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0b84d1e5-0641-4bb9-9da6-02f88938099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:11.336276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84d1e5-0641-4bb9-9da6-02f88938099e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:11.412676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:11.455790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74285eb7-8e23-4c4f-83d3-cd08aad97807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:11.456399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285eb7-8e23-4c4f-83d3-cd08aad97807 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:11.456732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285eb7-8e23-4c4f-83d3-cd08aad97807 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:11.457135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285eb7-8e23-4c4f-83d3-cd08aad97807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:11.458820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-74285eb7-8e23-4c4f-83d3-cd08aad97807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:11.552931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74285eb7-8e23-4c4f-83d3-cd08aad97807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:11.554576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74285eb7-8e23-4c4f-83d3-cd08aad97807 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:11.573750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 685/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:13.688038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85933-8f38-4035-ab19-77891eacf42a None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:13.688585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85933-8f38-4035-ab19-77891eacf42a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:13.688585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85933-8f38-4035-ab19-77891eacf42a None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:13.689871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 676/1361] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:13.695778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc20b7-08ea-46ce-abac-e192fb17b29e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:13.696158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc20b7-08ea-46ce-abac-e192fb17b29e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:13.696337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc20b7-08ea-46ce-abac-e192fb17b29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:13.780829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f9cc20b7-08ea-46ce-abac-e192fb17b29e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:13.782145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f9cc20b7-08ea-46ce-abac-e192fb17b29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:13.823654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:13.902132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 686/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:14.226796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85933-8f38-4035-ab19-77891eacf42a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:14.231226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b0a85933-8f38-4035-ab19-77891eacf42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:14.390002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85933-8f38-4035-ab19-77891eacf42a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:14.400532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:14.523888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d309-fe3d-4499-881a-14b63a5deca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:14.525050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d309-fe3d-4499-881a-14b63a5deca5 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:14.525501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d309-fe3d-4499-881a-14b63a5deca5 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:14.525941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d309-fe3d-4499-881a-14b63a5deca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:14.527887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a6d8d309-fe3d-4499-881a-14b63a5deca5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:14.632636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d309-fe3d-4499-881a-14b63a5deca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:14.634448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d309-fe3d-4499-881a-14b63a5deca5 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:14.655755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 677/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:16.900147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1baa-0834-493c-95ef-04c5e4fc5240 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:16.901113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1baa-0834-493c-95ef-04c5e4fc5240 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:16.901637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1baa-0834-493c-95ef-04c5e4fc5240 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:16.903275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 687/1364] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:16.909361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aed37-1574-4293-ba3e-c06cfaf169e1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:16.909832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aed37-1574-4293-ba3e-c06cfaf169e1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:16.909964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aed37-1574-4293-ba3e-c06cfaf169e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:16.927380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:16.987541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e98aed37-1574-4293-ba3e-c06cfaf169e1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:16.989279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e98aed37-1574-4293-ba3e-c06cfaf169e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:17.100143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 678/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:17.388050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6878f-bd95-4fab-9bf5-787d242a9fc3 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:17.388657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6878f-bd95-4fab-9bf5-787d242a9fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:17.388946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6878f-bd95-4fab-9bf5-787d242a9fc3 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:17.390327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 688/1366] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 06:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:17.396290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e2a04-65e8-42f5-ace8-5cdcd4d29116 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:17.396661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e2a04-65e8-42f5-ace8-5cdcd4d29116 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:17.397009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e2a04-65e8-42f5-ace8-5cdcd4d29116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:17.480681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d45e2a04-65e8-42f5-ace8-5cdcd4d29116 None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:17.481801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d45e2a04-65e8-42f5-ace8-5cdcd4d29116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:17.522278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:17.613296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 679/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 06:36:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:17.621704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6878f-bd95-4fab-9bf5-787d242a9fc3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:17.623397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8ef6878f-bd95-4fab-9bf5-787d242a9fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:17.789973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6878f-bd95-4fab-9bf5-787d242a9fc3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:17.811316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:17.915988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed6b43-7603-4286-9214-9f9d191d6a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:17.916706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6b43-7603-4286-9214-9f9d191d6a86 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:17.916948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6b43-7603-4286-9214-9f9d191d6a86 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:17.917121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6b43-7603-4286-9214-9f9d191d6a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:17.918776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cbed6b43-7603-4286-9214-9f9d191d6a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:18.012520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed6b43-7603-4286-9214-9f9d191d6a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:18.014538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed6b43-7603-4286-9214-9f9d191d6a86 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:18.033460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 689/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:18.654203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f58a-b33c-4e89-83c1-fef5eab1a92f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:18.655345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f58a-b33c-4e89-83c1-fef5eab1a92f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:18.655703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f58a-b33c-4e89-83c1-fef5eab1a92f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:18.657822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 680/1369] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:18.665093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ed1b2-6d0f-4067-ba1c-d0d240bcf291 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:18.665608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ed1b2-6d0f-4067-ba1c-d0d240bcf291 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:18.665889 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ed1b2-6d0f-4067-ba1c-d0d240bcf291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:18.701449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:18.740282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-224ed1b2-6d0f-4067-ba1c-d0d240bcf291 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:18.741520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-224ed1b2-6d0f-4067-ba1c-d0d240bcf291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:18.855471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 690/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:19.403572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8ef8-6848-4a69-93a5-45a7ea0b405d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:19.403861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8ef8-6848-4a69-93a5-45a7ea0b405d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:19.404136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8ef8-6848-4a69-93a5-45a7ea0b405d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:19.405672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 681/1371] 10.222.0.52 () {54 vars in 964 bytes} [Wed Aug 24 06:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:19.410806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a530348-50e5-4472-84a5-00099e661983 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:19.411361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a530348-50e5-4472-84a5-00099e661983 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:19.411361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a530348-50e5-4472-84a5-00099e661983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:19.490281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3a530348-50e5-4472-84a5-00099e661983 None None] MFA Rules not processed for user `c216522efef4447caac245392d95b985`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:19.491692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3a530348-50e5-4472-84a5-00099e661983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:19.542022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:19.575686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 691/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Aug 24 06:36:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:36:19.583926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b8ef8-6848-4a69-93a5-45a7ea0b405d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:19.585493 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a20b8ef8-6848-4a69-93a5-45a7ea0b405d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:19.696955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b8ef8-6848-4a69-93a5-45a7ea0b405d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:19.741504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:19.779921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a32e75-e19a-40da-8694-7c4eab3b669f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:19.781247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a32e75-e19a-40da-8694-7c4eab3b669f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:19.781991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a32e75-e19a-40da-8694-7c4eab3b669f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:19.782103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a32e75-e19a-40da-8694-7c4eab3b669f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:19.783765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a1a32e75-e19a-40da-8694-7c4eab3b669f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:19.940160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a32e75-e19a-40da-8694-7c4eab3b669f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:19.941734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a32e75-e19a-40da-8694-7c4eab3b669f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:19.963333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 682/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:20.030524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3adda-6443-4c80-87f0-0c2e30572d14 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:20.030825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3adda-6443-4c80-87f0-0c2e30572d14 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:20.031112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3adda-6443-4c80-87f0-0c2e30572d14 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:20.032527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 692/1374] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:20.038776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5affd-78e4-4830-b7e1-1f4480cbbb1c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:20.039297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5affd-78e4-4830-b7e1-1f4480cbbb1c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:20.039950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5affd-78e4-4830-b7e1-1f4480cbbb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:20.131181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dcc5affd-78e4-4830-b7e1-1f4480cbbb1c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:20.132843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dcc5affd-78e4-4830-b7e1-1f4480cbbb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:20.157088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:20.257598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 683/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:20.549074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3adda-6443-4c80-87f0-0c2e30572d14 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:20.550901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-73c3adda-6443-4c80-87f0-0c2e30572d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:20.707359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3adda-6443-4c80-87f0-0c2e30572d14 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:20.759370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:20.824953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf79185-d75c-4208-8d4b-4135cccd4385 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:20.825862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79185-d75c-4208-8d4b-4135cccd4385 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:20.826178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79185-d75c-4208-8d4b-4135cccd4385 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:20.826455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79185-d75c-4208-8d4b-4135cccd4385 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:20.828187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-caf79185-d75c-4208-8d4b-4135cccd4385 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:20.929206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf79185-d75c-4208-8d4b-4135cccd4385 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:20.931116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf79185-d75c-4208-8d4b-4135cccd4385 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:20.950590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 693/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:22.893710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba4274-94f3-4d9b-a458-2b248d83a730 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:22.894298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba4274-94f3-4d9b-a458-2b248d83a730 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:22.894298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba4274-94f3-4d9b-a458-2b248d83a730 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:22.895758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 684/1377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:22.901370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ded6d1-3642-46f4-8582-515559a8eb60 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:22.901832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ded6d1-3642-46f4-8582-515559a8eb60 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:22.901941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ded6d1-3642-46f4-8582-515559a8eb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:22.982744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-17ded6d1-3642-46f4-8582-515559a8eb60 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:22.984737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-17ded6d1-3642-46f4-8582-515559a8eb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:22.992182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:23.102969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 694/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:23.392125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba4274-94f3-4d9b-a458-2b248d83a730 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:23.394078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-25ba4274-94f3-4d9b-a458-2b248d83a730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:23.575125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba4274-94f3-4d9b-a458-2b248d83a730 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:23.589485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:23.709507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfe6b4-4073-48bf-b176-68d1061622fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:23.710037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfe6b4-4073-48bf-b176-68d1061622fa service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:23.710315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfe6b4-4073-48bf-b176-68d1061622fa service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:23.710591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfe6b4-4073-48bf-b176-68d1061622fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:23.712223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-40cfe6b4-4073-48bf-b176-68d1061622fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:23.808909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfe6b4-4073-48bf-b176-68d1061622fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:23.810607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfe6b4-4073-48bf-b176-68d1061622fa service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:23.830180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 685/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:25.795844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a43d3c-2285-43df-9b6a-2f57d4dc5260 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:25.796329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a43d3c-2285-43df-9b6a-2f57d4dc5260 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:25.796329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a43d3c-2285-43df-9b6a-2f57d4dc5260 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:25.797923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 695/1380] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:25.803746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87f1b-ad6b-4f6c-aa70-3319808afc00 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:25.804254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87f1b-ad6b-4f6c-aa70-3319808afc00 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:25.804556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87f1b-ad6b-4f6c-aa70-3319808afc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:25.817825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:25.888151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b1f87f1b-ad6b-4f6c-aa70-3319808afc00 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:25.889404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b1f87f1b-ad6b-4f6c-aa70-3319808afc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:25.996851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 686/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:26.288634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a43d3c-2285-43df-9b6a-2f57d4dc5260 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:26.290179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41a43d3c-2285-43df-9b6a-2f57d4dc5260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:26.463462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:26.476675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a43d3c-2285-43df-9b6a-2f57d4dc5260 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:26.614349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fce6b-2d08-4164-8f15-154537e5ec10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:26.615052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fce6b-2d08-4164-8f15-154537e5ec10 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:26.615339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fce6b-2d08-4164-8f15-154537e5ec10 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:26.615591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fce6b-2d08-4164-8f15-154537e5ec10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:26.617272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ba6fce6b-2d08-4164-8f15-154537e5ec10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:26.726448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fce6b-2d08-4164-8f15-154537e5ec10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:26.728130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fce6b-2d08-4164-8f15-154537e5ec10 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:26.740650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:26.751974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 696/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:28.755006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183611f6-338d-40df-b9d2-28becddeb0b0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:28.755674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183611f6-338d-40df-b9d2-28becddeb0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:28.755674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183611f6-338d-40df-b9d2-28becddeb0b0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:28.757114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 687/1383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:28.763376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443813d2-1d00-4f2c-b6cc-81e93f0e127a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:28.763830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443813d2-1d00-4f2c-b6cc-81e93f0e127a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:28.764159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443813d2-1d00-4f2c-b6cc-81e93f0e127a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:28.848292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-443813d2-1d00-4f2c-b6cc-81e93f0e127a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:28.849910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-443813d2-1d00-4f2c-b6cc-81e93f0e127a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:28.956541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 697/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:36:29.315215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183611f6-338d-40df-b9d2-28becddeb0b0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:29.317959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-183611f6-338d-40df-b9d2-28becddeb0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:29.413943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:29.482489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183611f6-338d-40df-b9d2-28becddeb0b0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:29.620457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b208f-0d58-4a4a-968f-8a5dada16d32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:29.621293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b208f-0d58-4a4a-968f-8a5dada16d32 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:29.621655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b208f-0d58-4a4a-968f-8a5dada16d32 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:29.622010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b208f-0d58-4a4a-968f-8a5dada16d32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:29.624173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6d6b208f-0d58-4a4a-968f-8a5dada16d32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:29.686948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:29.741732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b208f-0d58-4a4a-968f-8a5dada16d32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:29.743444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b208f-0d58-4a4a-968f-8a5dada16d32 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:29.764398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 688/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:32.540755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb367-3e9d-4f91-8c3b-ba0313bdfc6b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:32.541518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb367-3e9d-4f91-8c3b-ba0313bdfc6b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:32.541518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb367-3e9d-4f91-8c3b-ba0313bdfc6b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:32.542780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 698/1386] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:36:32.549117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a1aa5-0a00-4534-a6ed-8ab1ea385555 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:32.549725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a1aa5-0a00-4534-a6ed-8ab1ea385555 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:32.550042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a1aa5-0a00-4534-a6ed-8ab1ea385555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:32.634829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-853a1aa5-0a00-4534-a6ed-8ab1ea385555 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:32.636210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-853a1aa5-0a00-4534-a6ed-8ab1ea385555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:32.760762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 689/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:33.057250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bb367-3e9d-4f91-8c3b-ba0313bdfc6b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:33.059573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3a3bb367-3e9d-4f91-8c3b-ba0313bdfc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:33.145714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:33.225597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bb367-3e9d-4f91-8c3b-ba0313bdfc6b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:33.344786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc390554-7b4b-4423-b0e0-49da1e8b9d5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:33.345311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc390554-7b4b-4423-b0e0-49da1e8b9d5d service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:33.345585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc390554-7b4b-4423-b0e0-49da1e8b9d5d service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:33.345854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc390554-7b4b-4423-b0e0-49da1e8b9d5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:33.347548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fc390554-7b4b-4423-b0e0-49da1e8b9d5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:33.387756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:33.443774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc390554-7b4b-4423-b0e0-49da1e8b9d5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:33.445425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc390554-7b4b-4423-b0e0-49da1e8b9d5d service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:33.463978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 699/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:35.001880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695e417-86f9-44f7-9088-3ec54e935d81 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.002459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695e417-86f9-44f7-9088-3ec54e935d81 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.002459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695e417-86f9-44f7-9088-3ec54e935d81 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.003898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 690/1389] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:35.009295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46afdf-b5b4-4bf9-ab2e-8394e18cfefb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.009774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46afdf-b5b4-4bf9-ab2e-8394e18cfefb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.009880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46afdf-b5b4-4bf9-ab2e-8394e18cfefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.078124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1b46afdf-b5b4-4bf9-ab2e-8394e18cfefb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:35.079377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1b46afdf-b5b4-4bf9-ab2e-8394e18cfefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.184554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 700/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:36:35.395444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37d13e-9d6b-47e3-911d-799ce5634b71 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.395669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37d13e-9d6b-47e3-911d-799ce5634b71 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.395903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37d13e-9d6b-47e3-911d-799ce5634b71 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.397321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 691/1391] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:35.402966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.403346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.403626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.420265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:35.475431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:35.476670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.496200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5ef3a-0d7c-4040-9681-554dd47d9d7b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.496581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5ef3a-0d7c-4040-9681-554dd47d9d7b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.496933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5ef3a-0d7c-4040-9681-554dd47d9d7b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.498378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 692/1392] 10.222.0.52 () {54 vars in 967 bytes} [Wed Aug 24 06:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:35.503405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.503695 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.504001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.536075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:35.582897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] MFA Rules not processed for user `380786eb9b7848d59f3bb0007726dd2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:35.583649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 701/1393] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:35.587344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.664312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 693/1394] 10.222.0.52 () {58 vars in 1064 bytes} [Wed Aug 24 06:36:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:35.672027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:35.673600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.774669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cda00-2fd0-4afb-a285-10d6e7d28a64 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:35.783294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:35.853468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e27a00-f591-4bd4-951e-f0c61d9c70df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380786eb9b7848d59f3bb0007726dd2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:35.854081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e27a00-f591-4bd4-951e-f0c61d9c70df service glance] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:35.854356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e27a00-f591-4bd4-951e-f0c61d9c70df service glance] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:35.854622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e27a00-f591-4bd4-951e-f0c61d9c70df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:35.856268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-82e27a00-f591-4bd4-951e-f0c61d9c70df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.883944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:35.885174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:35.972495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e27a00-f591-4bd4-951e-f0c61d9c70df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:35.973802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e27a00-f591-4bd4-951e-f0c61d9c70df service glance] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:35.991695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 702/1395] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:36.008197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486df9b-705a-4c90-a6aa-73832cc05f84 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:36.011795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:36.131304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7ec34-4770-4728-a62f-23c255d1b04a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:36.131936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7ec34-4770-4728-a62f-23c255d1b04a service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:36.132218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7ec34-4770-4728-a62f-23c255d1b04a service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:36.132471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7ec34-4770-4728-a62f-23c255d1b04a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:36.134115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-41a7ec34-4770-4728-a62f-23c255d1b04a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:36.235006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7ec34-4770-4728-a62f-23c255d1b04a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:36.236721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7ec34-4770-4728-a62f-23c255d1b04a service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:36.256946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 694/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:36.762202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e9047-7b4c-41a4-94b4-bd68650722fd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:36.762676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e9047-7b4c-41a4-94b4-bd68650722fd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:36.762787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e9047-7b4c-41a4-94b4-bd68650722fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:36.810947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:36.835693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1d1e9047-7b4c-41a4-94b4-bd68650722fd None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:36.836940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1d1e9047-7b4c-41a4-94b4-bd68650722fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:36.939231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 703/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:36:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:36.984640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814abc-6bcf-410d-8d5c-19ee48866741 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:36.985033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814abc-6bcf-410d-8d5c-19ee48866741 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:36.985407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814abc-6bcf-410d-8d5c-19ee48866741 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:36.986985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 695/1398] 10.222.0.52 () {54 vars in 943 bytes} [Wed Aug 24 06:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:36.994325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f99c57-9df2-4f97-984d-0e38a7c621e8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:36.994885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f99c57-9df2-4f97-984d-0e38a7c621e8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:36.995309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f99c57-9df2-4f97-984d-0e38a7c621e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:37.078526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-64f99c57-9df2-4f97-984d-0e38a7c621e8 None None] MFA Rules not processed for user `f7ca3818552143069b0bacbb5bec7b62`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:37.080253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-64f99c57-9df2-4f97-984d-0e38a7c621e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:37.146207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:37.197047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 704/1399] 10.222.0.52 () {58 vars in 1040 bytes} [Wed Aug 24 06:36:36 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:37.207422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814abc-6bcf-410d-8d5c-19ee48866741 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:37.209007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-82814abc-6bcf-410d-8d5c-19ee48866741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:37.219866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:37.354821 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814abc-6bcf-410d-8d5c-19ee48866741 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:37.461928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:37.469500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832be3d-e21b-42a9-80c8-d349dd4034d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:37.470115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832be3d-e21b-42a9-80c8-d349dd4034d5 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:37.470399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832be3d-e21b-42a9-80c8-d349dd4034d5 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:37.470669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832be3d-e21b-42a9-80c8-d349dd4034d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:37.473007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e832be3d-e21b-42a9-80c8-d349dd4034d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:37.563043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832be3d-e21b-42a9-80c8-d349dd4034d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:37.564978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832be3d-e21b-42a9-80c8-d349dd4034d5 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:37.573896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 696/1400] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:36:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:36:38.242733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8db36-241a-41dd-ad7f-b00aa29ee451 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:38.243313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8db36-241a-41dd-ad7f-b00aa29ee451 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:38.243504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8db36-241a-41dd-ad7f-b00aa29ee451 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:38.245031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 705/1401] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:38.250625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c19eb-bf08-49a0-bb22-9426caebaf94 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:38.251049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c19eb-bf08-49a0-bb22-9426caebaf94 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:38.251333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c19eb-bf08-49a0-bb22-9426caebaf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:38.325828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fd8c19eb-bf08-49a0-bb22-9426caebaf94 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:38.327042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fd8c19eb-bf08-49a0-bb22-9426caebaf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:38.415367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:38.444224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 697/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:38.737914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8db36-241a-41dd-ad7f-b00aa29ee451 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:38.739574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-85c8db36-241a-41dd-ad7f-b00aa29ee451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:38.911948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8db36-241a-41dd-ad7f-b00aa29ee451 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:38.967464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:39.082472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cfc2a-1458-44c0-b820-4b01b00fe358 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:39.082850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cfc2a-1458-44c0-b820-4b01b00fe358 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:39.083116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cfc2a-1458-44c0-b820-4b01b00fe358 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:39.083369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cfc2a-1458-44c0-b820-4b01b00fe358 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:39.085024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-409cfc2a-1458-44c0-b820-4b01b00fe358 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:39.186298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cfc2a-1458-44c0-b820-4b01b00fe358 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:39.188346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cfc2a-1458-44c0-b820-4b01b00fe358 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:39.207913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 706/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:41.199812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37c2f-e23c-4346-92a1-353f65bd5587 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:41.200438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37c2f-e23c-4346-92a1-353f65bd5587 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:41.200438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37c2f-e23c-4346-92a1-353f65bd5587 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:41.205721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 698/1404] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:41.213779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ed7d-456d-46a2-81fb-ed9bb71d413f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:41.214767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ed7d-456d-46a2-81fb-ed9bb71d413f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:41.215116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259ed7d-456d-46a2-81fb-ed9bb71d413f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:41.286148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:41.296033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c259ed7d-456d-46a2-81fb-ed9bb71d413f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:41.297410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c259ed7d-456d-46a2-81fb-ed9bb71d413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:41.407322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 707/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:41.698257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37c2f-e23c-4346-92a1-353f65bd5587 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:41.700002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-84a37c2f-e23c-4346-92a1-353f65bd5587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:41.862295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37c2f-e23c-4346-92a1-353f65bd5587 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:41.929284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:41.992211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f65ca-111d-4e78-b1e4-e461b22f2e49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:41.993130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f65ca-111d-4e78-b1e4-e461b22f2e49 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:41.993417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f65ca-111d-4e78-b1e4-e461b22f2e49 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:41.993698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f65ca-111d-4e78-b1e4-e461b22f2e49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:41.995363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a20f65ca-111d-4e78-b1e4-e461b22f2e49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:42.099215 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f65ca-111d-4e78-b1e4-e461b22f2e49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:42.101222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f65ca-111d-4e78-b1e4-e461b22f2e49 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:42.120674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 699/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:44.108003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eec53-24e7-46c9-8c77-d70cd4e56d5e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:44.108583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eec53-24e7-46c9-8c77-d70cd4e56d5e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:44.108583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eec53-24e7-46c9-8c77-d70cd4e56d5e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:44.110121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 708/1407] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:44.116011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561f9d5-8fef-41cd-9af0-6e41e6178448 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:44.117232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561f9d5-8fef-41cd-9af0-6e41e6178448 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:44.117582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561f9d5-8fef-41cd-9af0-6e41e6178448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:44.198837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a561f9d5-8fef-41cd-9af0-6e41e6178448 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:44.200283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a561f9d5-8fef-41cd-9af0-6e41e6178448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:44.236794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:44.319778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 700/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:44.613013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eec53-24e7-46c9-8c77-d70cd4e56d5e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:44.614718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-679eec53-24e7-46c9-8c77-d70cd4e56d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:44.752972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:44.783874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eec53-24e7-46c9-8c77-d70cd4e56d5e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:44.913572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248ff3b-64b0-42d7-9861-f2b29b48a00e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:44.914104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248ff3b-64b0-42d7-9861-f2b29b48a00e service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:44.914389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248ff3b-64b0-42d7-9861-f2b29b48a00e service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:44.914664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248ff3b-64b0-42d7-9861-f2b29b48a00e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:44.916342 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e248ff3b-64b0-42d7-9861-f2b29b48a00e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:45.014970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:45.019047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248ff3b-64b0-42d7-9861-f2b29b48a00e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:45.022128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248ff3b-64b0-42d7-9861-f2b29b48a00e service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:45.044552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 709/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:54.317550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=33 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:55.039855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:59.987494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce01cc-441b-40d6-ad00-2a53e327e146 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:59.988133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce01cc-441b-40d6-ad00-2a53e327e146 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:59.988133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce01cc-441b-40d6-ad00-2a53e327e146 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:59.989570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 701/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Aug 24 06:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:59.995165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8fc48-c2fa-4aad-9e24-0f1f1a2637cc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:59.995635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8fc48-c2fa-4aad-9e24-0f1f1a2637cc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:59.995754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8fc48-c2fa-4aad-9e24-0f1f1a2637cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:00.079056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1ca8fc48-c2fa-4aad-9e24-0f1f1a2637cc None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:00.080318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1ca8fc48-c2fa-4aad-9e24-0f1f1a2637cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:00.202754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 710/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 24 06:36:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:00.222132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42631360-4642-4082-a4ff-232fdf231397 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:00.222451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42631360-4642-4082-a4ff-232fdf231397 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:00.222765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42631360-4642-4082-a4ff-232fdf231397 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:00.224536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 702/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Aug 24 06:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:00.231252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd12111-0e4f-47c5-b362-c883e88160ce None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:00.231571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd12111-0e4f-47c5-b362-c883e88160ce None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:00.231849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd12111-0e4f-47c5-b362-c883e88160ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:00.316943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:00.321854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7cd12111-0e4f-47c5-b362-c883e88160ce None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:00.324189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7cd12111-0e4f-47c5-b362-c883e88160ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:00.448537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 711/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 24 06:37:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:02.853842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48904bb1-f138-430e-888f-0ca12bd09ae9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:02.854414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48904bb1-f138-430e-888f-0ca12bd09ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:02.854414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48904bb1-f138-430e-888f-0ca12bd09ae9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:02.856614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:02.862501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5af18-6b8e-4771-85c7-cdb000f698d7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:02.863003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5af18-6b8e-4771-85c7-cdb000f698d7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:02.863003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5af18-6b8e-4771-85c7-cdb000f698d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:02.932617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c9c5af18-6b8e-4771-85c7-cdb000f698d7 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:02.933838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c9c5af18-6b8e-4771-85c7-cdb000f698d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:02.995351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:03.045064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:03.360017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48904bb1-f138-430e-888f-0ca12bd09ae9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:03.362795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-48904bb1-f138-430e-888f-0ca12bd09ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:03.535933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48904bb1-f138-430e-888f-0ca12bd09ae9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:03.631030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:03.676670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6f0e8-bd50-4435-95e8-7ee6ae0fad99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:03.677289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6f0e8-bd50-4435-95e8-7ee6ae0fad99 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:03.677574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6f0e8-bd50-4435-95e8-7ee6ae0fad99 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:03.677848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6f0e8-bd50-4435-95e8-7ee6ae0fad99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:03.679517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a4a6f0e8-bd50-4435-95e8-7ee6ae0fad99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:03.775766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6f0e8-bd50-4435-95e8-7ee6ae0fad99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:03.777385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6f0e8-bd50-4435-95e8-7ee6ae0fad99 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:03.796005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 704/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:04.707259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070882-c0e9-4ab1-b188-4b23b60832d2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:04.708314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070882-c0e9-4ab1-b188-4b23b60832d2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:04.708688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070882-c0e9-4ab1-b188-4b23b60832d2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:04.710823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:04.719270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a04d-b3ec-4696-bd14-141d2a914004 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:04.719982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a04d-b3ec-4696-bd14-141d2a914004 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:04.720158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a04d-b3ec-4696-bd14-141d2a914004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:04.815957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6e94a04d-b3ec-4696-bd14-141d2a914004 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:04.817906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6e94a04d-b3ec-4696-bd14-141d2a914004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:04.850473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:04.922982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:05.076176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9070882-c0e9-4ab1-b188-4b23b60832d2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:05.078641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c9070882-c0e9-4ab1-b188-4b23b60832d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:05.246817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9070882-c0e9-4ab1-b188-4b23b60832d2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:05.291236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:05.374805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a516a-5b4b-410d-9cc1-c66afd923905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:05.375403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a516a-5b4b-410d-9cc1-c66afd923905 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:05.375687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a516a-5b4b-410d-9cc1-c66afd923905 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:05.375960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a516a-5b4b-410d-9cc1-c66afd923905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:05.377589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-447a516a-5b4b-410d-9cc1-c66afd923905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:05.471034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a516a-5b4b-410d-9cc1-c66afd923905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:05.472696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a516a-5b4b-410d-9cc1-c66afd923905 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:05.492019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 714/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:06.392711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc6cc6-df0a-4e3d-b065-46cdb3464d7f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:06.393570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc6cc6-df0a-4e3d-b065-46cdb3464d7f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:06.393570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc6cc6-df0a-4e3d-b065-46cdb3464d7f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:06.396131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:06.401560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815857df-5184-48da-887f-03bb0f9769d9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:06.402034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815857df-5184-48da-887f-03bb0f9769d9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:06.402155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815857df-5184-48da-887f-03bb0f9769d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:06.459824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:06.483506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-815857df-5184-48da-887f-03bb0f9769d9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:06.484671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-815857df-5184-48da-887f-03bb0f9769d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:06.603768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:06.760178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc6cc6-df0a-4e3d-b065-46cdb3464d7f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:06.762781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-16dc6cc6-df0a-4e3d-b065-46cdb3464d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:06.933502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc6cc6-df0a-4e3d-b065-46cdb3464d7f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:06.946566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:07.055302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9363bb-9492-4462-9e43-a56094596098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:07.055838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9363bb-9492-4462-9e43-a56094596098 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:07.056098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9363bb-9492-4462-9e43-a56094596098 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:07.056364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9363bb-9492-4462-9e43-a56094596098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:07.058001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3b9363bb-9492-4462-9e43-a56094596098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:07.158895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9363bb-9492-4462-9e43-a56094596098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:07.160559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9363bb-9492-4462-9e43-a56094596098 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:07.179794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 707/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:08.072019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023331e-9346-4b5f-98f7-90b06791156a None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:08.072639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023331e-9346-4b5f-98f7-90b06791156a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:08.072639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023331e-9346-4b5f-98f7-90b06791156a None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:08.073963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:08.080185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d944c-336f-45ea-a333-6b9c8a64bcf7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:08.080665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d944c-336f-45ea-a333-6b9c8a64bcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:08.080758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d944c-336f-45ea-a333-6b9c8a64bcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:08.107016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:08.165978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-087d944c-336f-45ea-a333-6b9c8a64bcf7 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:08.167178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-087d944c-336f-45ea-a333-6b9c8a64bcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:08.282729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:08.448013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023331e-9346-4b5f-98f7-90b06791156a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:08.450091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6023331e-9346-4b5f-98f7-90b06791156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:08.586747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:08.625089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023331e-9346-4b5f-98f7-90b06791156a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:08.748631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd9261-5ae8-4d6f-b592-d3538341e839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:08.749245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd9261-5ae8-4d6f-b592-d3538341e839 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:08.749525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd9261-5ae8-4d6f-b592-d3538341e839 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:08.749797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd9261-5ae8-4d6f-b592-d3538341e839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:08.751492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-10fd9261-5ae8-4d6f-b592-d3538341e839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:08.844357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:08.859351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd9261-5ae8-4d6f-b592-d3538341e839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:08.861010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd9261-5ae8-4d6f-b592-d3538341e839 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:08.880771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 717/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:09.758517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92898d0-0e8c-4ebb-b3e0-6a8072846221 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:09.759133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92898d0-0e8c-4ebb-b3e0-6a8072846221 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:09.759133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92898d0-0e8c-4ebb-b3e0-6a8072846221 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:09.760455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:09.766828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0b1a4-e920-4666-8dbe-fa994344b224 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:09.767310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0b1a4-e920-4666-8dbe-fa994344b224 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:09.767445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0b1a4-e920-4666-8dbe-fa994344b224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:09.856961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d5a0b1a4-e920-4666-8dbe-fa994344b224 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:09.858196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d5a0b1a4-e920-4666-8dbe-fa994344b224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:09.969362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:10.129940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92898d0-0e8c-4ebb-b3e0-6a8072846221 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:10.131982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c92898d0-0e8c-4ebb-b3e0-6a8072846221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:10.229665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:10.295884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92898d0-0e8c-4ebb-b3e0-6a8072846221 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:10.411353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a96b-0c42-4b9d-baf1-f3841786484b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:10.411839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a96b-0c42-4b9d-baf1-f3841786484b service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:10.412116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a96b-0c42-4b9d-baf1-f3841786484b service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:10.412385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a96b-0c42-4b9d-baf1-f3841786484b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:10.414176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-20b9a96b-0c42-4b9d-baf1-f3841786484b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:10.468183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:10.512322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a96b-0c42-4b9d-baf1-f3841786484b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:10.513947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a96b-0c42-4b9d-baf1-f3841786484b service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:10.532385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 710/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:11.359528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3d88-3972-40ab-a33a-194e6333441b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:11.360481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3d88-3972-40ab-a33a-194e6333441b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:11.360481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3d88-3972-40ab-a33a-194e6333441b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:11.362675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:11.369726 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c92292-042f-47c2-8afd-8884edad4455 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:11.370167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c92292-042f-47c2-8afd-8884edad4455 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:11.370280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c92292-042f-47c2-8afd-8884edad4455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:11.472375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-28c92292-042f-47c2-8afd-8884edad4455 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:11.473542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-28c92292-042f-47c2-8afd-8884edad4455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:11.580879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:11.732929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3d88-3972-40ab-a33a-194e6333441b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:11.735183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-47ee3d88-3972-40ab-a33a-194e6333441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:11.799112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:11.903612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3d88-3972-40ab-a33a-194e6333441b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:12.022414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b9319-d997-44ff-8480-0963f64b7ecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:12.023314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b9319-d997-44ff-8480-0963f64b7ecf service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:12.023700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b9319-d997-44ff-8480-0963f64b7ecf service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:12.024070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b9319-d997-44ff-8480-0963f64b7ecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:12.026662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b71b9319-d997-44ff-8480-0963f64b7ecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:12.037534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:12.126777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b9319-d997-44ff-8480-0963f64b7ecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:12.128448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b9319-d997-44ff-8480-0963f64b7ecf service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:12.148168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 720/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:12.982178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e26e7e-43a0-49e7-bacb-9acc27464af4 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:12.982735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e26e7e-43a0-49e7-bacb-9acc27464af4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:12.982735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e26e7e-43a0-49e7-bacb-9acc27464af4 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:12.984747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:12.992543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e319-b93f-4940-8e1e-7594adf13c48 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:12.993054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e319-b93f-4940-8e1e-7594adf13c48 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:12.993169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e319-b93f-4940-8e1e-7594adf13c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:13.091903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8737e319-b93f-4940-8e1e-7594adf13c48 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:13.093134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8737e319-b93f-4940-8e1e-7594adf13c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:13.166761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:13.200465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:37:13.350317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e26e7e-43a0-49e7-bacb-9acc27464af4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:13.351868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d9e26e7e-43a0-49e7-bacb-9acc27464af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:13.371090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:13.508713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e26e7e-43a0-49e7-bacb-9acc27464af4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:13.606132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:13.636211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fcc9f-4fd5-4ea3-af05-86ec0e3e1e1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:13.636822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fcc9f-4fd5-4ea3-af05-86ec0e3e1e1c service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:13.637226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fcc9f-4fd5-4ea3-af05-86ec0e3e1e1c service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:13.637501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fcc9f-4fd5-4ea3-af05-86ec0e3e1e1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:13.639131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8c2fcc9f-4fd5-4ea3-af05-86ec0e3e1e1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:13.745231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fcc9f-4fd5-4ea3-af05-86ec0e3e1e1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:13.747743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fcc9f-4fd5-4ea3-af05-86ec0e3e1e1c service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:13.775328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 713/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:14.590949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84b7c7-5334-48ec-9b73-51239505b1b5 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:14.591920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84b7c7-5334-48ec-9b73-51239505b1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:14.591920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84b7c7-5334-48ec-9b73-51239505b1b5 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:14.593994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:14.602722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5928-6fdc-43e2-af1a-43086ea8c15e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:14.603231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5928-6fdc-43e2-af1a-43086ea8c15e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:14.603454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5928-6fdc-43e2-af1a-43086ea8c15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:14.693197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b7bd5928-6fdc-43e2-af1a-43086ea8c15e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:14.694448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b7bd5928-6fdc-43e2-af1a-43086ea8c15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:14.740352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:14.813043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:14.967170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84b7c7-5334-48ec-9b73-51239505b1b5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:14.968652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fd84b7c7-5334-48ec-9b73-51239505b1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:15.149803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84b7c7-5334-48ec-9b73-51239505b1b5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:15.216188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:15.291041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f923311-52e5-4563-bf97-57eeee1ae52a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:15.291692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923311-52e5-4563-bf97-57eeee1ae52a service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:15.291976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923311-52e5-4563-bf97-57eeee1ae52a service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:15.292248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f923311-52e5-4563-bf97-57eeee1ae52a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:15.293884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0f923311-52e5-4563-bf97-57eeee1ae52a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:15.391410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f923311-52e5-4563-bf97-57eeee1ae52a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:15.393053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f923311-52e5-4563-bf97-57eeee1ae52a service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:15.413486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 723/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:16.220495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d49502-57af-44eb-98f5-fe843a4e8e9b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:16.221013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d49502-57af-44eb-98f5-fe843a4e8e9b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:16.221013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d49502-57af-44eb-98f5-fe843a4e8e9b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:16.222394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:16.228588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33cfae-c7a2-4b3b-aac9-d66933b71345 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:16.229082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33cfae-c7a2-4b3b-aac9-d66933b71345 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:16.229189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33cfae-c7a2-4b3b-aac9-d66933b71345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:16.316074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5b33cfae-c7a2-4b3b-aac9-d66933b71345 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:16.317833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5b33cfae-c7a2-4b3b-aac9-d66933b71345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:16.320043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:16.431739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:16.583975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d49502-57af-44eb-98f5-fe843a4e8e9b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:16.586238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d1d49502-57af-44eb-98f5-fe843a4e8e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:16.745875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d49502-57af-44eb-98f5-fe843a4e8e9b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:16.774351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:16.866422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f67ae4-cc3e-4de7-91e9-351ca7656a71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:16.866960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f67ae4-cc3e-4de7-91e9-351ca7656a71 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:16.867269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f67ae4-cc3e-4de7-91e9-351ca7656a71 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:16.867579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f67ae4-cc3e-4de7-91e9-351ca7656a71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:16.869199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-77f67ae4-cc3e-4de7-91e9-351ca7656a71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:16.965331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f67ae4-cc3e-4de7-91e9-351ca7656a71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:16.967060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f67ae4-cc3e-4de7-91e9-351ca7656a71 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:16.987338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 716/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:18.864392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb458931-d0dc-4625-9aa3-8b7cb9f3e7c6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:18.865013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb458931-d0dc-4625-9aa3-8b7cb9f3e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:18.865013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb458931-d0dc-4625-9aa3-8b7cb9f3e7c6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:18.866419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:18.874886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29cf79-b824-487c-9cf2-33dffd777011 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:18.875693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29cf79-b824-487c-9cf2-33dffd777011 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:18.875872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29cf79-b824-487c-9cf2-33dffd777011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:18.924181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:18.951768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ea29cf79-b824-487c-9cf2-33dffd777011 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:18.953035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ea29cf79-b824-487c-9cf2-33dffd777011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:19.065306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:19.349886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb458931-d0dc-4625-9aa3-8b7cb9f3e7c6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:19.351504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bb458931-d0dc-4625-9aa3-8b7cb9f3e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:19.469149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb458931-d0dc-4625-9aa3-8b7cb9f3e7c6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:19.503833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:19.550243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34129b5f-9ab3-4548-aa33-07c348fcb3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:19.550898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129b5f-9ab3-4548-aa33-07c348fcb3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:19.551177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129b5f-9ab3-4548-aa33-07c348fcb3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:19.551458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129b5f-9ab3-4548-aa33-07c348fcb3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:19.553230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-34129b5f-9ab3-4548-aa33-07c348fcb3e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:19.673200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34129b5f-9ab3-4548-aa33-07c348fcb3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:19.674657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34129b5f-9ab3-4548-aa33-07c348fcb3e9 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:19.693148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 726/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:21.748454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db21e-40b9-4ae8-860c-7568ecbf46ed None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:21.749391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db21e-40b9-4ae8-860c-7568ecbf46ed None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:21.749391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db21e-40b9-4ae8-860c-7568ecbf46ed None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:21.751661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:21.757129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f90cb-39e3-44d6-bb82-403b775b6af1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:21.757592 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f90cb-39e3-44d6-bb82-403b775b6af1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:21.757689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f90cb-39e3-44d6-bb82-403b775b6af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:21.826662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8d8f90cb-39e3-44d6-bb82-403b775b6af1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:21.827922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8d8f90cb-39e3-44d6-bb82-403b775b6af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:21.841149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:21.934481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:22.220988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295db21e-40b9-4ae8-860c-7568ecbf46ed None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:22.222554 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-295db21e-40b9-4ae8-860c-7568ecbf46ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:22.342325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295db21e-40b9-4ae8-860c-7568ecbf46ed None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:22.356142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:22.432151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c826f-b0c6-41f3-85e7-44a4f0f27cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:22.432774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c826f-b0c6-41f3-85e7-44a4f0f27cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:22.433064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c826f-b0c6-41f3-85e7-44a4f0f27cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:22.433341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c826f-b0c6-41f3-85e7-44a4f0f27cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:22.434950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0d6c826f-b0c6-41f3-85e7-44a4f0f27cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:22.554818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c826f-b0c6-41f3-85e7-44a4f0f27cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:22.556318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c826f-b0c6-41f3-85e7-44a4f0f27cc6 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:22.575932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 719/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:24.914808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e8be1-8e71-421e-bbe7-db8a81b5a308 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:24.915740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e8be1-8e71-421e-bbe7-db8a81b5a308 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:24.915740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e8be1-8e71-421e-bbe7-db8a81b5a308 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:24.917918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:37:24.926586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3c7b4-6df0-4391-87f2-ff0ce2ada565 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:24.927056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3c7b4-6df0-4391-87f2-ff0ce2ada565 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:24.927177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3c7b4-6df0-4391-87f2-ff0ce2ada565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:24.978724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:24.998135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d0c3c7b4-6df0-4391-87f2-ff0ce2ada565 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:24.999384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d0c3c7b4-6df0-4391-87f2-ff0ce2ada565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:25.111525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:37:25.403850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e8be1-8e71-421e-bbe7-db8a81b5a308 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:25.406350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-887e8be1-8e71-421e-bbe7-db8a81b5a308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:25.518450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e8be1-8e71-421e-bbe7-db8a81b5a308 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:25.559711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:25.594960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74013d3-1603-4b4d-aedd-bd83554b2f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:25.595611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74013d3-1603-4b4d-aedd-bd83554b2f96 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:25.595886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74013d3-1603-4b4d-aedd-bd83554b2f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:25.596162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74013d3-1603-4b4d-aedd-bd83554b2f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:25.597779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b74013d3-1603-4b4d-aedd-bd83554b2f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:25.713283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74013d3-1603-4b4d-aedd-bd83554b2f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:25.714746 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74013d3-1603-4b4d-aedd-bd83554b2f96 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:25.733267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 729/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:27.804391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f523a1-4c2d-477e-8579-b4e3b64a8bce None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:27.805321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f523a1-4c2d-477e-8579-b4e3b64a8bce None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:27.805321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f523a1-4c2d-477e-8579-b4e3b64a8bce None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:27.807489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:27.813110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0e539-a84c-4f69-8103-17bd21f36b41 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:27.813545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0e539-a84c-4f69-8103-17bd21f36b41 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:27.813669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0e539-a84c-4f69-8103-17bd21f36b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:27.882331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3fb0e539-a84c-4f69-8103-17bd21f36b41 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:27.883677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3fb0e539-a84c-4f69-8103-17bd21f36b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:27.907228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:28.000532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:28.285004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f523a1-4c2d-477e-8579-b4e3b64a8bce None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:28.286875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-15f523a1-4c2d-477e-8579-b4e3b64a8bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:28.396952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f523a1-4c2d-477e-8579-b4e3b64a8bce None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:28.417448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:28.475429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396f458-7bae-4892-9b91-312e42efcf6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:28.475960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396f458-7bae-4892-9b91-312e42efcf6a service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:28.476237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396f458-7bae-4892-9b91-312e42efcf6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:28.476513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396f458-7bae-4892-9b91-312e42efcf6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:28.478105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9396f458-7bae-4892-9b91-312e42efcf6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:28.592259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396f458-7bae-4892-9b91-312e42efcf6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:28.593706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396f458-7bae-4892-9b91-312e42efcf6a service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:28.612205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 722/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:30.601202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e8b8-dfa8-4132-8884-ac1e56bdf233 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:30.602125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e8b8-dfa8-4132-8884-ac1e56bdf233 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:30.602125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584e8b8-dfa8-4132-8884-ac1e56bdf233 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:30.604496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:30.614051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b689f84-aebc-4748-a35d-bb1ac19081c1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:30.614790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b689f84-aebc-4748-a35d-bb1ac19081c1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:30.615320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b689f84-aebc-4748-a35d-bb1ac19081c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:30.691692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:30.694986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8b689f84-aebc-4748-a35d-bb1ac19081c1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:30.696923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b689f84-aebc-4748-a35d-bb1ac19081c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:30.800136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:31.086632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e8b8-dfa8-4132-8884-ac1e56bdf233 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:31.089013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c584e8b8-dfa8-4132-8884-ac1e56bdf233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:31.224716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584e8b8-dfa8-4132-8884-ac1e56bdf233 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:31.277998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:31.307110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34812ba-0f12-4e81-b2de-02ef68862638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:31.307752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34812ba-0f12-4e81-b2de-02ef68862638 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:31.308035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34812ba-0f12-4e81-b2de-02ef68862638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:31.308309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34812ba-0f12-4e81-b2de-02ef68862638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:31.309923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d34812ba-0f12-4e81-b2de-02ef68862638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:31.425746 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34812ba-0f12-4e81-b2de-02ef68862638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:31.427509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34812ba-0f12-4e81-b2de-02ef68862638 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:31.447944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 732/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:33.491366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486715b1-7de0-4261-90d0-df835eb098f3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:33.492255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486715b1-7de0-4261-90d0-df835eb098f3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:33.492255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486715b1-7de0-4261-90d0-df835eb098f3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:33.494300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:33.499981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1bdc7-5aa3-4fe9-b76f-b6e0c9ff4aa3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:33.500435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1bdc7-5aa3-4fe9-b76f-b6e0c9ff4aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:33.500542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1bdc7-5aa3-4fe9-b76f-b6e0c9ff4aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:33.573730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6cd1bdc7-5aa3-4fe9-b76f-b6e0c9ff4aa3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:33.574969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6cd1bdc7-5aa3-4fe9-b76f-b6e0c9ff4aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:33.607348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:33.683374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:37:33.973288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486715b1-7de0-4261-90d0-df835eb098f3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:33.976705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-486715b1-7de0-4261-90d0-df835eb098f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:34.113127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486715b1-7de0-4261-90d0-df835eb098f3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:34.141983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:34.192810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde6a22-ee6f-49fc-8d1e-87cd3c7f3aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:34.193432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6a22-ee6f-49fc-8d1e-87cd3c7f3aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:34.193710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6a22-ee6f-49fc-8d1e-87cd3c7f3aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:34.193980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde6a22-ee6f-49fc-8d1e-87cd3c7f3aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:34.195613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4cde6a22-ee6f-49fc-8d1e-87cd3c7f3aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:34.313789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde6a22-ee6f-49fc-8d1e-87cd3c7f3aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:34.315248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde6a22-ee6f-49fc-8d1e-87cd3c7f3aa2 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:34.335362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 725/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:36.284615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ccc2f-7d57-4f43-b1e3-70d0920dd8d6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:36.285194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ccc2f-7d57-4f43-b1e3-70d0920dd8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:36.285194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ccc2f-7d57-4f43-b1e3-70d0920dd8d6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:36.286707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:36.292326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba5d5-ee29-4e36-b805-b38976e26dbe None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:36.292871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba5d5-ee29-4e36-b805-b38976e26dbe None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:36.292871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eba5d5-ee29-4e36-b805-b38976e26dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:36.362991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f5eba5d5-ee29-4e36-b805-b38976e26dbe None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:36.364376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f5eba5d5-ee29-4e36-b805-b38976e26dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:36.373016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:36.477414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:36.760665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ccc2f-7d57-4f43-b1e3-70d0920dd8d6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:36.762705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0a7ccc2f-7d57-4f43-b1e3-70d0920dd8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:36.884339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ccc2f-7d57-4f43-b1e3-70d0920dd8d6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:36.943775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:36.963516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a4a42-b95f-4f26-843e-cd58b4cc20b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:36.964160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a4a42-b95f-4f26-843e-cd58b4cc20b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:36.964445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a4a42-b95f-4f26-843e-cd58b4cc20b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:36.964721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a4a42-b95f-4f26-843e-cd58b4cc20b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:36.966364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4a3a4a42-b95f-4f26-843e-cd58b4cc20b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:37.096988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a4a42-b95f-4f26-843e-cd58b4cc20b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:37.098462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a4a42-b95f-4f26-843e-cd58b4cc20b1 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:37.118574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 735/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:39.088824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58398acb-d4ec-4945-8ea3-f77d7447b08b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:39.089897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58398acb-d4ec-4945-8ea3-f77d7447b08b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:39.089897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58398acb-d4ec-4945-8ea3-f77d7447b08b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:39.091894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:39.100581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bee8e3-55bb-41c2-82db-753199c4de35 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:39.101387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bee8e3-55bb-41c2-82db-753199c4de35 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:39.101704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bee8e3-55bb-41c2-82db-753199c4de35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:39.178543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d0bee8e3-55bb-41c2-82db-753199c4de35 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:39.180258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d0bee8e3-55bb-41c2-82db-753199c4de35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:39.226727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:39.288929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:39.578940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58398acb-d4ec-4945-8ea3-f77d7447b08b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:39.580567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-58398acb-d4ec-4945-8ea3-f77d7447b08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:39.694858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58398acb-d4ec-4945-8ea3-f77d7447b08b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:39.874647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:39.919443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158448f-aa91-4e2b-856e-374727153bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:39.920051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158448f-aa91-4e2b-856e-374727153bfa service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:39.920331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158448f-aa91-4e2b-856e-374727153bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:39.920600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158448f-aa91-4e2b-856e-374727153bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:39.922191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0158448f-aa91-4e2b-856e-374727153bfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:40.054179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158448f-aa91-4e2b-856e-374727153bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:40.055634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158448f-aa91-4e2b-856e-374727153bfa service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:40.074662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 728/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:42.032811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb054d-b5df-47df-b739-967bac331c8f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:42.033807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb054d-b5df-47df-b739-967bac331c8f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:42.033807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb054d-b5df-47df-b739-967bac331c8f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:42.036091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:42.041806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cacd47-b66b-4ddb-b2ff-4e8dc00f4fc6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:42.042304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cacd47-b66b-4ddb-b2ff-4e8dc00f4fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:42.042410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cacd47-b66b-4ddb-b2ff-4e8dc00f4fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:42.116762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-59cacd47-b66b-4ddb-b2ff-4e8dc00f4fc6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:42.118011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-59cacd47-b66b-4ddb-b2ff-4e8dc00f4fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:42.138807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:42.222771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:42.515230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb054d-b5df-47df-b739-967bac331c8f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:42.516822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-49eb054d-b5df-47df-b739-967bac331c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:42.633421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb054d-b5df-47df-b739-967bac331c8f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:42.731296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:42.742183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8c5a1-4025-47e9-94b6-decc362fc0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:42.743016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8c5a1-4025-47e9-94b6-decc362fc0aa service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:42.743441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8c5a1-4025-47e9-94b6-decc362fc0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:42.743768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8c5a1-4025-47e9-94b6-decc362fc0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:42.745947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5aa8c5a1-4025-47e9-94b6-decc362fc0aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:42.882525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8c5a1-4025-47e9-94b6-decc362fc0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:42.884102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8c5a1-4025-47e9-94b6-decc362fc0aa service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:42.903688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 738/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:44.952577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c0d4-3415-4fbf-99b0-583fab359aa0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:44.953462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c0d4-3415-4fbf-99b0-583fab359aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:44.953462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c0d4-3415-4fbf-99b0-583fab359aa0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:44.955717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:37:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:37:44.962824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc74a3-a5d2-4299-93bf-e271c765d833 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:44.963335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc74a3-a5d2-4299-93bf-e271c765d833 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:44.963447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc74a3-a5d2-4299-93bf-e271c765d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:45.036974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8bcc74a3-a5d2-4299-93bf-e271c765d833 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:37:45.038233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8bcc74a3-a5d2-4299-93bf-e271c765d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:45.087961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:45.147357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:37:45.439594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06c0d4-3415-4fbf-99b0-583fab359aa0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:37:45.442304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fb06c0d4-3415-4fbf-99b0-583fab359aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:45.568624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06c0d4-3415-4fbf-99b0-583fab359aa0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:37:45.621371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:45.648548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e0260-68c3-4a98-8077-aa8caac885b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:37:45.649158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e0260-68c3-4a98-8077-aa8caac885b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:37:45.649435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e0260-68c3-4a98-8077-aa8caac885b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:37:45.649700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e0260-68c3-4a98-8077-aa8caac885b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:37:45.651335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-116e0260-68c3-4a98-8077-aa8caac885b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:37:45.765612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e0260-68c3-4a98-8077-aa8caac885b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:37:45.767067 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e0260-68c3-4a98-8077-aa8caac885b3 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:37:45.786908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 731/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:37:55.145220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=22 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:55.784155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=66 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:05.607232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6355ff-6345-4acb-bdb9-2918b854cb44 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:05.607814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6355ff-6345-4acb-bdb9-2918b854cb44 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:05.607814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6355ff-6345-4acb-bdb9-2918b854cb44 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:05.609977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 740/1471] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:38:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:05.617769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c266a-b02f-413d-8a6c-1203d33052dc None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:05.618188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c266a-b02f-413d-8a6c-1203d33052dc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:05.618320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c266a-b02f-413d-8a6c-1203d33052dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:05.696399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c65c266a-b02f-413d-8a6c-1203d33052dc None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:05.697739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c65c266a-b02f-413d-8a6c-1203d33052dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:05.803661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 732/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:06.100767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb728-3337-4e2d-a3e0-fe22e24f7f2c None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:06.101044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb728-3337-4e2d-a3e0-fe22e24f7f2c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:06.101318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb728-3337-4e2d-a3e0-fe22e24f7f2c None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:06.102678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 741/1473] 10.222.0.52 () {54 vars in 967 bytes} [Wed Aug 24 06:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:06.107715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c925-a706-4a55-950c-2afee85f36bd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:06.108005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c925-a706-4a55-950c-2afee85f36bd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:06.108311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c925-a706-4a55-950c-2afee85f36bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:06.187367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0a77c925-a706-4a55-950c-2afee85f36bd None None] MFA Rules not processed for user `380786eb9b7848d59f3bb0007726dd2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:06.188576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0a77c925-a706-4a55-950c-2afee85f36bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:06.209715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:06.268245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 733/1474] 10.222.0.52 () {58 vars in 1064 bytes} [Wed Aug 24 06:38:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:06.276116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb728-3337-4e2d-a3e0-fe22e24f7f2c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:06.277639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1e2cb728-3337-4e2d-a3e0-fe22e24f7f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:06.390657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb728-3337-4e2d-a3e0-fe22e24f7f2c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:06.471230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38e1d1-f029-4d6c-a2ad-eb2b79a8c994 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380786eb9b7848d59f3bb0007726dd2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:06.471882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38e1d1-f029-4d6c-a2ad-eb2b79a8c994 service glance] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:06.472169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38e1d1-f029-4d6c-a2ad-eb2b79a8c994 service glance] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:06.472671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38e1d1-f029-4d6c-a2ad-eb2b79a8c994 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:06.474331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0e38e1d1-f029-4d6c-a2ad-eb2b79a8c994 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:06.536539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:06.598870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38e1d1-f029-4d6c-a2ad-eb2b79a8c994 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:06.600427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38e1d1-f029-4d6c-a2ad-eb2b79a8c994 service glance] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:06.619850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 742/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:08.647414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b383935-42e0-4bc8-8224-aa9ea34faab4 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:08.648059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b383935-42e0-4bc8-8224-aa9ea34faab4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:08.648059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b383935-42e0-4bc8-8224-aa9ea34faab4 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:08.649348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 734/1476] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:08.656012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a91f-7296-49c2-9e4f-7683fdfd19e4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:08.656693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a91f-7296-49c2-9e4f-7683fdfd19e4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:08.656867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a91f-7296-49c2-9e4f-7683fdfd19e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:08.740354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0953a91f-7296-49c2-9e4f-7683fdfd19e4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:08.741572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0953a91f-7296-49c2-9e4f-7683fdfd19e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:08.843574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 743/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:09.110181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b383935-42e0-4bc8-8224-aa9ea34faab4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:09.111760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6b383935-42e0-4bc8-8224-aa9ea34faab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:09.266782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b383935-42e0-4bc8-8224-aa9ea34faab4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:09.324387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:09.371081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01735762-6454-44f2-b40d-01551b9f8c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:09.371643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01735762-6454-44f2-b40d-01551b9f8c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:09.371923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01735762-6454-44f2-b40d-01551b9f8c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:09.372194 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01735762-6454-44f2-b40d-01551b9f8c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:09.372841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01735762-6454-44f2-b40d-01551b9f8c1e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:09.374589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-01735762-6454-44f2-b40d-01551b9f8c1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:38:09.374589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:38:09.374589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:38:09.375765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01735762-6454-44f2-b40d-01551b9f8c1e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:09.394708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 735/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Aug 24 06:38:09 2022] GET /identity/v3/projects => generated 2553 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:38:11.336018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba70778-ebca-468b-be53-44856830400c None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:11.336627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba70778-ebca-468b-be53-44856830400c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:11.336627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba70778-ebca-468b-be53-44856830400c None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:11.337967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 744/1479] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:11.344516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834818-c659-4b07-bb9d-9d2e17a13153 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:11.345319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834818-c659-4b07-bb9d-9d2e17a13153 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:11.345477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834818-c659-4b07-bb9d-9d2e17a13153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:11.425026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3a834818-c659-4b07-bb9d-9d2e17a13153 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:11.426725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3a834818-c659-4b07-bb9d-9d2e17a13153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:11.542967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 736/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:11.843448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba70778-ebca-468b-be53-44856830400c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:11.845974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7ba70778-ebca-468b-be53-44856830400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:11.857248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:12.004811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba70778-ebca-468b-be53-44856830400c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:12.089381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:12.128180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c8879-d17b-40ee-8209-e2574bb8c1bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:12.128839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8879-d17b-40ee-8209-e2574bb8c1bb service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:12.129099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8879-d17b-40ee-8209-e2574bb8c1bb service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:12.129366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8879-d17b-40ee-8209-e2574bb8c1bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:12.131011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fc8c8879-d17b-40ee-8209-e2574bb8c1bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:12.242408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c8879-d17b-40ee-8209-e2574bb8c1bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:12.244059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c8879-d17b-40ee-8209-e2574bb8c1bb service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:12.263532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 745/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:14.380339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c278-bf96-48f7-ba1a-3a1243bd4a6d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:14.381280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c278-bf96-48f7-ba1a-3a1243bd4a6d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:14.381280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c278-bf96-48f7-ba1a-3a1243bd4a6d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:14.383570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 737/1482] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:38:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:14.390539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5c14-67db-4ed3-854f-1818e9f50c35 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:14.391248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5c14-67db-4ed3-854f-1818e9f50c35 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:14.391443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c5c14-67db-4ed3-854f-1818e9f50c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:14.469032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-924c5c14-67db-4ed3-854f-1818e9f50c35 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:14.470176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-924c5c14-67db-4ed3-854f-1818e9f50c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:14.523473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:14.590672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 746/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:14.876203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939c278-bf96-48f7-ba1a-3a1243bd4a6d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:14.878860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f939c278-bf96-48f7-ba1a-3a1243bd4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:14.903637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:14.997451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939c278-bf96-48f7-ba1a-3a1243bd4a6d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:15.076593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012fe62-acb5-4a1b-942d-2a414f64b7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:15.077206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012fe62-acb5-4a1b-942d-2a414f64b7ba service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:15.077485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012fe62-acb5-4a1b-942d-2a414f64b7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:15.077756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012fe62-acb5-4a1b-942d-2a414f64b7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:15.079374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e012fe62-acb5-4a1b-942d-2a414f64b7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:15.154296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:15.198854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012fe62-acb5-4a1b-942d-2a414f64b7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:15.200348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012fe62-acb5-4a1b-942d-2a414f64b7ba service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:15.219898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 738/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:17.251838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54965779-2ac5-4a4b-b7ee-62c88cba447c None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:17.252401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54965779-2ac5-4a4b-b7ee-62c88cba447c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:17.252401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54965779-2ac5-4a4b-b7ee-62c88cba447c None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:17.254376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 747/1485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:17.260929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2f839-5b38-475b-ad13-cc0f3380648e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:17.261693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2f839-5b38-475b-ad13-cc0f3380648e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:17.261833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2f839-5b38-475b-ad13-cc0f3380648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:17.339998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-25b2f839-5b38-475b-ad13-cc0f3380648e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:17.341224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-25b2f839-5b38-475b-ad13-cc0f3380648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:17.453367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 739/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:17.755728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54965779-2ac5-4a4b-b7ee-62c88cba447c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:17.757320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-54965779-2ac5-4a4b-b7ee-62c88cba447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:17.873044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54965779-2ac5-4a4b-b7ee-62c88cba447c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:17.951076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:17.953095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc07d3-a2b4-4c27-9f71-fa89e64506ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:17.953847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc07d3-a2b4-4c27-9f71-fa89e64506ab service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:17.954342 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc07d3-a2b4-4c27-9f71-fa89e64506ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:17.954636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc07d3-a2b4-4c27-9f71-fa89e64506ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:17.956723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-39cc07d3-a2b4-4c27-9f71-fa89e64506ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:18.072278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc07d3-a2b4-4c27-9f71-fa89e64506ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:18.073674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc07d3-a2b4-4c27-9f71-fa89e64506ab service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:18.092857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 748/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:20.152790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bcef0-80c3-4242-b96f-836073f9584a None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:20.153418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bcef0-80c3-4242-b96f-836073f9584a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:20.153418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bcef0-80c3-4242-b96f-836073f9584a None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:20.154714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 740/1488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:20.160823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe4e3-874e-4238-8104-6ea1c8e3cfb9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:20.161344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe4e3-874e-4238-8104-6ea1c8e3cfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:20.161449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe4e3-874e-4238-8104-6ea1c8e3cfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:20.235032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c1abe4e3-874e-4238-8104-6ea1c8e3cfb9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:20.236306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c1abe4e3-874e-4238-8104-6ea1c8e3cfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:20.298867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:20.350533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 749/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:20.645463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bcef0-80c3-4242-b96f-836073f9584a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:20.647061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b91bcef0-80c3-4242-b96f-836073f9584a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:20.688051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:20.777044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bcef0-80c3-4242-b96f-836073f9584a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:20.859108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4f4c4-9ad2-41d2-8d01-d4e59afe8393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:20.859764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4f4c4-9ad2-41d2-8d01-d4e59afe8393 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:20.860052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4f4c4-9ad2-41d2-8d01-d4e59afe8393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:20.860326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4f4c4-9ad2-41d2-8d01-d4e59afe8393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:20.861945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-34a4f4c4-9ad2-41d2-8d01-d4e59afe8393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:20.955703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:20.985825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4f4c4-9ad2-41d2-8d01-d4e59afe8393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:20.987280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4f4c4-9ad2-41d2-8d01-d4e59afe8393 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:21.005091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 741/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:23.111313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb011a-7ddf-4a08-8f99-c8b49784d404 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:23.111929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb011a-7ddf-4a08-8f99-c8b49784d404 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:23.111929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb011a-7ddf-4a08-8f99-c8b49784d404 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:23.113273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 750/1491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Aug 24 06:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:23.121073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31abb4-6185-4e76-8ed9-8a7316a715ec None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:23.121859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31abb4-6185-4e76-8ed9-8a7316a715ec None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:23.122035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31abb4-6185-4e76-8ed9-8a7316a715ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:23.198897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4c31abb4-6185-4e76-8ed9-8a7316a715ec None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:23.200139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4c31abb4-6185-4e76-8ed9-8a7316a715ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:23.307028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 742/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Aug 24 06:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:23.598628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb011a-7ddf-4a08-8f99-c8b49784d404 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:23.601587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8dbb011a-7ddf-4a08-8f99-c8b49784d404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:23.860950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb011a-7ddf-4a08-8f99-c8b49784d404 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:23.945580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a351ddf-8f3c-48f8-80a7-274399fafb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:23.946198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a351ddf-8f3c-48f8-80a7-274399fafb31 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:23.946476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a351ddf-8f3c-48f8-80a7-274399fafb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:23.946732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a351ddf-8f3c-48f8-80a7-274399fafb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:23.948374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0a351ddf-8f3c-48f8-80a7-274399fafb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:23.957996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:24.076438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a351ddf-8f3c-48f8-80a7-274399fafb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:24.077889 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a351ddf-8f3c-48f8-80a7-274399fafb31 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:24.096463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 751/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:33.304776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=82 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:34.094082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=50 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:40.522889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:40.523903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:40.523903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:40.610196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:39:40.611450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:40.714419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 743/1494] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:39:40.723049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de632ae-5635-4787-8479-a88bab529a6b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:40.723616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de632ae-5635-4787-8479-a88bab529a6b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:40.723616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de632ae-5635-4787-8479-a88bab529a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:40.800588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8de632ae-5635-4787-8479-a88bab529a6b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:39:40.801706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8de632ae-5635-4787-8479-a88bab529a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:40.916781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 752/1495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:39:40.921980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:40.923278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:40.935573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fff1d-edc8-4623-817a-9fd8a2b00846 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:41.015453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:41.057953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:41.058483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:41.058768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:41.059037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:41.059729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:39:41.060037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:41.061094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:41.061094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:39:41.061094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:39:41.061972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:41.074051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 744/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:39:41.083371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de632ae-5635-4787-8479-a88bab529a6b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:41.092632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de632ae-5635-4787-8479-a88bab529a6b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:41.165230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:41.204290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:41.204898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:41.205172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:41.205442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:41.206080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:41.207882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:41.207882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Aug 24 06:39:41.207882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: Aug 24 06:39:41.209065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:41.268146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 753/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:39:41 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:39:41.272655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:41.274202 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:41.308736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83451152-ed4b-483f-aaf0-cfc6b32d935f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:41.424482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:41.425224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:41.425531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:41.425802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:41.426449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:41.428820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:41.428820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:39:41.428820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:39:41.430730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:41.487975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 745/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:39:41 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:39:41.492458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:41.494057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:41.559535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e2581-235c-4e53-a91a-511e9baf4d95 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:41.650937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:41.662684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:41.663325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:41.663603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:41.663875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:41.664520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:41.665541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:41.665541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Aug 24 06:39:41.665541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: Aug 24 06:39:41.666536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:41.679229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 754/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:39:41.683562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:41.685094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:41.697368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5d768-0faa-4b1c-9ce8-90a299f3c7df admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:41.723378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:41.803240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acede4-b8e6-4505-9b2f-1f50293c3d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:41.803842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acede4-b8e6-4505-9b2f-1f50293c3d79 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:41.804118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acede4-b8e6-4505-9b2f-1f50293c3d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:41.804394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acede4-b8e6-4505-9b2f-1f50293c3d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:41.805041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acede4-b8e6-4505-9b2f-1f50293c3d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:41.806057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-09acede4-b8e6-4505-9b2f-1f50293c3d79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:41.806057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:39:41.806057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:39:41.806948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acede4-b8e6-4505-9b2f-1f50293c3d79 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:41.820994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 746/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:39:41.825640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:41.827223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:41.872102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ed998-4cd4-43b9-a13c-fea0fbf47f1e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:41.975514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86172c62-ea9c-42fe-87a8-77d9a1dbf36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:41.976190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172c62-ea9c-42fe-87a8-77d9a1dbf36f admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:41.976475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172c62-ea9c-42fe-87a8-77d9a1dbf36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:41.976745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172c62-ea9c-42fe-87a8-77d9a1dbf36f admin admin] PATH_INFO: `/v3/projects/67d6f77be0d9486abbdebd32a57d4bc1/users/673c68d828d54c7ea187a5ce0fd5f371/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:42.000170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86172c62-ea9c-42fe-87a8-77d9a1dbf36f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d6f77be0d9486abbdebd32a57d4bc1, user_id=673c68d828d54c7ea187a5ce0fd5f371, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:42.001242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-86172c62-ea9c-42fe-87a8-77d9a1dbf36f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:42.001242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Aug 24 06:39:42.001242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: Aug 24 06:39:42.003284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86172c62-ea9c-42fe-87a8-77d9a1dbf36f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:42.029676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 755/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:39:41 2022] PUT /identity/v3/projects/67d6f77be0d9486abbdebd32a57d4bc1/users/673c68d828d54c7ea187a5ce0fd5f371/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:39:42.042560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea850ed-c05c-454c-ab22-eecdc9cd9ba5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:42.042862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea850ed-c05c-454c-ab22-eecdc9cd9ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:42.043133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea850ed-c05c-454c-ab22-eecdc9cd9ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:42.104042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4ea850ed-c05c-454c-ab22-eecdc9cd9ba5 None None] MFA Rules not processed for user `673c68d828d54c7ea187a5ce0fd5f371`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:39:42.105180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ea850ed-c05c-454c-ab22-eecdc9cd9ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:42.193241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 747/1502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:39:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:39:42.208013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e397-8d85-4729-b2cf-4b95fdaa29e8 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:42.208359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e397-8d85-4729-b2cf-4b95fdaa29e8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:42.208659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e397-8d85-4729-b2cf-4b95fdaa29e8 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:42.210044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 756/1503] 10.222.0.52 () {54 vars in 964 bytes} [Wed Aug 24 06:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:39:42.216155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ac930-0c8e-4644-a5c7-8c1fd3ad932d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:42.216600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ac930-0c8e-4644-a5c7-8c1fd3ad932d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:42.217042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ac930-0c8e-4644-a5c7-8c1fd3ad932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:42.255977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:42.303455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f02ac930-0c8e-4644-a5c7-8c1fd3ad932d None None] MFA Rules not processed for user `c216522efef4447caac245392d95b985`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:39:42.304668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f02ac930-0c8e-4644-a5c7-8c1fd3ad932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:42.385780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 748/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Aug 24 06:39:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:39:42.393832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e397-8d85-4729-b2cf-4b95fdaa29e8 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:42.395401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-60c1e397-8d85-4729-b2cf-4b95fdaa29e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:42.454381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:42.518030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e397-8d85-4729-b2cf-4b95fdaa29e8 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:42.604271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:42.604895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:42.605171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:42.605442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:42.607038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:42.714169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:42.715673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:42.719162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:42.739094 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 757/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:39:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:39:42.784432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55effd7-77a5-44ab-b87a-5de57c87dc55 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:42.784915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55effd7-77a5-44ab-b87a-5de57c87dc55 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:42.785044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55effd7-77a5-44ab-b87a-5de57c87dc55 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:42.786299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 749/1506] 10.222.0.52 () {52 vars in 830 bytes} [Wed Aug 24 06:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:39:42.790194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:42.791472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:42.900735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee76b54-3387-45bc-85e1-71458607417b service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:43.004907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:43.005537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:43.005813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:43.006081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] PATH_INFO: `/v3/users/673c68d828d54c7ea187a5ce0fd5f371` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:43.020001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=673c68d828d54c7ea187a5ce0fd5f371)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:43.021029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:43.021029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Aug 24 06:39:43.021029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: Aug 24 06:39:43.022238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:43.031854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:43.097773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '673c68d828d54c7ea187a5ce0fd5f371'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:43.098246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '673c68d828d54c7ea187a5ce0fd5f371'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:43.114095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '673c68d828d54c7ea187a5ce0fd5f371'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:43.114657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '673c68d828d54c7ea187a5ce0fd5f371'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:43.126928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '673c68d828d54c7ea187a5ce0fd5f371'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:43.158454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 758/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:39:42 2022] DELETE /identity/v3/users/673c68d828d54c7ea187a5ce0fd5f371 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:39:43.163401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:43.163667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:43.163975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:43.223649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:39:43.224909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:43.313760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:43.329908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 750/1508] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:39:43.342074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:43.343964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:43.478832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784732c6-c7ce-4731-8c19-d26275b34eb4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:43.546690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:43.567859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f1932-9c9c-4050-a767-06bd1c86d1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:43.568472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1932-9c9c-4050-a767-06bd1c86d1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:43.568752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1932-9c9c-4050-a767-06bd1c86d1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:43.569021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f1932-9c9c-4050-a767-06bd1c86d1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:43.570620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e49f1932-9c9c-4050-a767-06bd1c86d1d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:43.683569 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f1932-9c9c-4050-a767-06bd1c86d1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:43.685074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f1932-9c9c-4050-a767-06bd1c86d1d4 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:43.703884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 759/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:39:44.055631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:39:44.057485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:39:44.081760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695757f6-24fc-4d65-adcd-735e40a2c488 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:39:44.192134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:39:44.192775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:39:44.193052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:39:44.193321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] PATH_INFO: `/v3/projects/67d6f77be0d9486abbdebd32a57d4bc1` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:39:44.203430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d6f77be0d9486abbdebd32a57d4bc1)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:39:44.204461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:39:44.204461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:39:44.204461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:39:44.205693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:39:44.295104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d6f77be0d9486abbdebd32a57d4bc1', 'request_id': 'req-e3efab5e-12af-4490-8d1a-b07a1df51fb9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:44.304183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d6f77be0d9486abbdebd32a57d4bc1', 'request_id': 'req-e3efab5e-12af-4490-8d1a-b07a1df51fb9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:44.326580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d6f77be0d9486abbdebd32a57d4bc1', 'request_id': 'req-e3efab5e-12af-4490-8d1a-b07a1df51fb9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:44.360290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] The token cache is being invalidate because project 67d6f77be0d9486abbdebd32a57d4bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:39:44.360561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:44.361122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e3efab5e-12af-4490-8d1a-b07a1df51fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:39:44.363612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 751/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:39:44 2022] DELETE /identity/v3/projects/67d6f77be0d9486abbdebd32a57d4bc1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:39:53.701489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=62 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:54.358197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:14.893086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80bdcc-3f75-466e-a996-ff1807f6341e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:14.893819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80bdcc-3f75-466e-a996-ff1807f6341e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:14.893819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80bdcc-3f75-466e-a996-ff1807f6341e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:14.895605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:14.903645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd44d10-0210-47ac-9eb2-e781c4c93371 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:14.904160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd44d10-0210-47ac-9eb2-e781c4c93371 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:14.904254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd44d10-0210-47ac-9eb2-e781c4c93371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:14.990874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4cd44d10-0210-47ac-9eb2-e781c4c93371 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:14.992110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4cd44d10-0210-47ac-9eb2-e781c4c93371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:15.116676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:15.276250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80bdcc-3f75-466e-a996-ff1807f6341e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:15.278743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6c80bdcc-3f75-466e-a996-ff1807f6341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:15.411764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80bdcc-3f75-466e-a996-ff1807f6341e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:15.417826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971393a-819d-4d2b-bfe2-1df4ff9fd1a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:15.418439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971393a-819d-4d2b-bfe2-1df4ff9fd1a0 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:15.418835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971393a-819d-4d2b-bfe2-1df4ff9fd1a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:15.419133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971393a-819d-4d2b-bfe2-1df4ff9fd1a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:15.428458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971393a-819d-4d2b-bfe2-1df4ff9fd1a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:15.430105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971393a-819d-4d2b-bfe2-1df4ff9fd1a0 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:15.441066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 761/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:16.270790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf214-6c6f-40c4-bf73-2df12ded200d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.271404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf214-6c6f-40c4-bf73-2df12ded200d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.271404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf214-6c6f-40c4-bf73-2df12ded200d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.272799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:16.278043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c27b18-6bed-4e7e-b81f-4a56486665fd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.278558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c27b18-6bed-4e7e-b81f-4a56486665fd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.278558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c27b18-6bed-4e7e-b81f-4a56486665fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.301158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-78c27b18-6bed-4e7e-b81f-4a56486665fd None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:16.302381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-78c27b18-6bed-4e7e-b81f-4a56486665fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:16.310941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:16.465295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf214-6c6f-40c4-bf73-2df12ded200d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:16.483935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf214-6c6f-40c4-bf73-2df12ded200d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:16.489634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a4ed0-0825-4a15-8892-a12053fd8d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:16.490238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4ed0-0825-4a15-8892-a12053fd8d06 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.490514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4ed0-0825-4a15-8892-a12053fd8d06 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.490791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4ed0-0825-4a15-8892-a12053fd8d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.499329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a4ed0-0825-4a15-8892-a12053fd8d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:16.501097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a4ed0-0825-4a15-8892-a12053fd8d06 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:16.511053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 754/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:16.970693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5b0a0-e79e-4515-9385-b64a430369f6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.970959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5b0a0-e79e-4515-9385-b64a430369f6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.971227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5b0a0-e79e-4515-9385-b64a430369f6 None None] PATH_INFO: `/v3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.972572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 763/1517] 10.222.0.58 () {54 vars in 925 bytes} [Wed Aug 24 06:41:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:16.979330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad77ac-c507-4303-8f42-a2b80c985437 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.979629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad77ac-c507-4303-8f42-a2b80c985437 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.979904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad77ac-c507-4303-8f42-a2b80c985437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:17.047558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-eaad77ac-c507-4303-8f42-a2b80c985437 None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:17.048702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eaad77ac-c507-4303-8f42-a2b80c985437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:17.067398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:17.134184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 755/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 24 06:41:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:20.959746 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108b539-3a6e-46ba-abe8-63671ca1235e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:20.960303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108b539-3a6e-46ba-abe8-63671ca1235e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:20.960303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108b539-3a6e-46ba-abe8-63671ca1235e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:20.961566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 764/1519] 10.222.0.52 () {54 vars in 961 bytes} [Wed Aug 24 06:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:20.967933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27daf031-4660-4de8-9c1b-a83576b05f38 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:20.968412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27daf031-4660-4de8-9c1b-a83576b05f38 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:20.968516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27daf031-4660-4de8-9c1b-a83576b05f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:20.990781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-27daf031-4660-4de8-9c1b-a83576b05f38 None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:20.992034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-27daf031-4660-4de8-9c1b-a83576b05f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:21.002767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 756/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Aug 24 06:41:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:21.010920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108b539-3a6e-46ba-abe8-63671ca1235e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:21.028225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108b539-3a6e-46ba-abe8-63671ca1235e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:21.033893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4781e6-49f8-4944-aadb-0edd6c373159 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:21.034480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4781e6-49f8-4944-aadb-0edd6c373159 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:21.034763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4781e6-49f8-4944-aadb-0edd6c373159 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:21.035034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4781e6-49f8-4944-aadb-0edd6c373159 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:21.044071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4781e6-49f8-4944-aadb-0edd6c373159 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:21.045658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4781e6-49f8-4944-aadb-0edd6c373159 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:21.055681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 765/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:27.758023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f2e76-f817-4417-8594-5893885d9f98 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:27.758984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f2e76-f817-4417-8594-5893885d9f98 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:27.758984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f2e76-f817-4417-8594-5893885d9f98 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:27.761174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:27.767068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2c232-873d-4dec-9279-101328431282 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:27.767750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2c232-873d-4dec-9279-101328431282 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:27.768085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2c232-873d-4dec-9279-101328431282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:27.788467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-14e2c232-873d-4dec-9279-101328431282 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:27.789672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-14e2c232-873d-4dec-9279-101328431282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:27.798075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:27.961765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f2e76-f817-4417-8594-5893885d9f98 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:27.978446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f2e76-f817-4417-8594-5893885d9f98 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:27.984433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd263d8-4a2d-4603-a031-0b15b4597eec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:27.985035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd263d8-4a2d-4603-a031-0b15b4597eec service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:27.985320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd263d8-4a2d-4603-a031-0b15b4597eec service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:27.985602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd263d8-4a2d-4603-a031-0b15b4597eec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:27.994197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd263d8-4a2d-4603-a031-0b15b4597eec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:27.995844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd263d8-4a2d-4603-a031-0b15b4597eec service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:28.005780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 758/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:28.843085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ff61b-1de6-4285-9f52-4f78fd5ecf04 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:28.843997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ff61b-1de6-4285-9f52-4f78fd5ecf04 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:28.844180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ff61b-1de6-4285-9f52-4f78fd5ecf04 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:28.846674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:41:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:28.851889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2d0b0-04b9-4326-9394-1b48b6fc7fd0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:28.852377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2d0b0-04b9-4326-9394-1b48b6fc7fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:28.852488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2d0b0-04b9-4326-9394-1b48b6fc7fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:28.873765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-81e2d0b0-04b9-4326-9394-1b48b6fc7fd0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:28.874990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-81e2d0b0-04b9-4326-9394-1b48b6fc7fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:28.885720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:29.046337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ff61b-1de6-4285-9f52-4f78fd5ecf04 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.068292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ff61b-1de6-4285-9f52-4f78fd5ecf04 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.076017 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c402982-1b02-40d5-a533-2c6106f09bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.076974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402982-1b02-40d5-a533-2c6106f09bfb service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.077451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402982-1b02-40d5-a533-2c6106f09bfb service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.077816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c402982-1b02-40d5-a533-2c6106f09bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.091868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c402982-1b02-40d5-a533-2c6106f09bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.094441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c402982-1b02-40d5-a533-2c6106f09bfb service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.105544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 768/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:33.002523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9c4db-3b5a-4788-ab3c-15ef012894f1 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:33.006847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9c4db-3b5a-4788-ab3c-15ef012894f1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:33.006847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9c4db-3b5a-4788-ab3c-15ef012894f1 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.006847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:41:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:33.012322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f1b00-ac8b-4670-a58a-97ff3fd84484 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:33.012809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f1b00-ac8b-4670-a58a-97ff3fd84484 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:33.012916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f1b00-ac8b-4670-a58a-97ff3fd84484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.035580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-848f1b00-ac8b-4670-a58a-97ff3fd84484 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:33.036810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-848f1b00-ac8b-4670-a58a-97ff3fd84484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:33.045966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:33.335230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9c4db-3b5a-4788-ab3c-15ef012894f1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:33.337386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bdd9c4db-3b5a-4788-ab3c-15ef012894f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:33.426688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9c4db-3b5a-4788-ab3c-15ef012894f1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:33.430658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef648a1-f4f9-46c1-91ee-0af6320f8f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:33.431301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef648a1-f4f9-46c1-91ee-0af6320f8f69 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:33.431589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef648a1-f4f9-46c1-91ee-0af6320f8f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:33.431871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef648a1-f4f9-46c1-91ee-0af6320f8f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.442238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef648a1-f4f9-46c1-91ee-0af6320f8f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:33.443731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef648a1-f4f9-46c1-91ee-0af6320f8f69 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:33.455977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 761/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:35.365726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf41a-f8a2-43d0-a071-46e98a38dae6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:35.366305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf41a-f8a2-43d0-a071-46e98a38dae6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:35.366305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bf41a-f8a2-43d0-a071-46e98a38dae6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:35.367708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:35.374173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448babae-d8f6-4b5a-b7c7-5e9efbeaa822 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:35.374891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448babae-d8f6-4b5a-b7c7-5e9efbeaa822 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:35.375064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448babae-d8f6-4b5a-b7c7-5e9efbeaa822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:35.403102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-448babae-d8f6-4b5a-b7c7-5e9efbeaa822 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:35.404372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-448babae-d8f6-4b5a-b7c7-5e9efbeaa822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:35.413798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:35.703108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bf41a-f8a2-43d0-a071-46e98a38dae6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:35.706083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8b7bf41a-f8a2-43d0-a071-46e98a38dae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:35.725962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bf41a-f8a2-43d0-a071-46e98a38dae6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:35.730251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed1b7e-def5-4224-9fd6-39053b57c938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:35.730859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed1b7e-def5-4224-9fd6-39053b57c938 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:35.731142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed1b7e-def5-4224-9fd6-39053b57c938 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:35.731442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed1b7e-def5-4224-9fd6-39053b57c938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:35.740953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed1b7e-def5-4224-9fd6-39053b57c938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:35.742413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed1b7e-def5-4224-9fd6-39053b57c938 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:35.752081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 771/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:39.165494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e138ef7-86c2-4db2-b463-851413b1b1fc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.166429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e138ef7-86c2-4db2-b463-851413b1b1fc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.166429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e138ef7-86c2-4db2-b463-851413b1b1fc None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.168817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:41:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:39.177727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05aba-0811-4368-a8c3-621b266da7b0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.178203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05aba-0811-4368-a8c3-621b266da7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.178310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d05aba-0811-4368-a8c3-621b266da7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.202885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-05d05aba-0811-4368-a8c3-621b266da7b0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:39.204162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-05d05aba-0811-4368-a8c3-621b266da7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.213729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:39.768445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e138ef7-86c2-4db2-b463-851413b1b1fc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.770715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9e138ef7-86c2-4db2-b463-851413b1b1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.854261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:39.868138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e138ef7-86c2-4db2-b463-851413b1b1fc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.871857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bbcd7-3bf1-4b8c-86f5-8678080f53df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.872474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bbcd7-3bf1-4b8c-86f5-8678080f53df service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.872756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bbcd7-3bf1-4b8c-86f5-8678080f53df service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.873015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bbcd7-3bf1-4b8c-86f5-8678080f53df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.881970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bbcd7-3bf1-4b8c-86f5-8678080f53df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.883452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bbcd7-3bf1-4b8c-86f5-8678080f53df service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.893344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 764/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:41.828595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38377b3-5a5e-4205-b267-d7e8d7b16d5d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.829209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38377b3-5a5e-4205-b267-d7e8d7b16d5d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.829209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38377b3-5a5e-4205-b267-d7e8d7b16d5d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.830838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:41.837392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128e88-729d-498d-81e0-9c9dd1e7baa4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.838035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128e88-729d-498d-81e0-9c9dd1e7baa4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.838210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128e88-729d-498d-81e0-9c9dd1e7baa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.861302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ab128e88-729d-498d-81e0-9c9dd1e7baa4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:41.862556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ab128e88-729d-498d-81e0-9c9dd1e7baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.871009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:42.426335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38377b3-5a5e-4205-b267-d7e8d7b16d5d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.429684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a38377b3-5a5e-4205-b267-d7e8d7b16d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:42.448459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38377b3-5a5e-4205-b267-d7e8d7b16d5d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.452041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901b60b-1c80-4dc4-bba3-1060fcfa233e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.452652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901b60b-1c80-4dc4-bba3-1060fcfa233e service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.452937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901b60b-1c80-4dc4-bba3-1060fcfa233e service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.453213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901b60b-1c80-4dc4-bba3-1060fcfa233e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.462965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901b60b-1c80-4dc4-bba3-1060fcfa233e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.464459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901b60b-1c80-4dc4-bba3-1060fcfa233e service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.474886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 774/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:51.856581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=11 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:52.469992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:05.368894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a8be-a38a-4f47-9e7a-d6558df2bd26 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.370280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a8be-a38a-4f47-9e7a-d6558df2bd26 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.371073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5a8be-a38a-4f47-9e7a-d6558df2bd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.408993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9cc5a8be-a38a-4f47-9e7a-d6558df2bd26 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.410601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9cc5a8be-a38a-4f47-9e7a-d6558df2bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.429969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 766/1540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.440468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ddff3-759b-4854-9a06-cefab8ae8c40 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.441841 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ddff3-759b-4854-9a06-cefab8ae8c40 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.441841 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ddff3-759b-4854-9a06-cefab8ae8c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.471131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b43ddff3-759b-4854-9a06-cefab8ae8c40 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.472326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b43ddff3-759b-4854-9a06-cefab8ae8c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.487259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 775/1541] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.493525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83464ca5-077a-4ff8-815e-a193891a69f7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.494118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83464ca5-077a-4ff8-815e-a193891a69f7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.494607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83464ca5-077a-4ff8-815e-a193891a69f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.499999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ddff3-759b-4854-9a06-cefab8ae8c40 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:05.509804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ddff3-759b-4854-9a06-cefab8ae8c40 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:05.526752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-83464ca5-077a-4ff8-815e-a193891a69f7 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.532073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-83464ca5-077a-4ff8-815e-a193891a69f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.543614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:05.545156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.546098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.548100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.550159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:05.550820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:05.553100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:43:05.553100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Aug 24 06:43:05.553100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: Aug 24 06:43:05.555509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ba92c-0e22-4254-82dc-d572221a96dc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:05.558569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 767/1542] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.560858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26783f-4c77-4416-b9af-a113f324a9d6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.561357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26783f-4c77-4416-b9af-a113f324a9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.561871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26783f-4c77-4416-b9af-a113f324a9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.580939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 776/1543] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:05.580939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43eb2b-ae6b-48b3-bdef-7c8209527a7f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.580939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43eb2b-ae6b-48b3-bdef-7c8209527a7f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.580939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43eb2b-ae6b-48b3-bdef-7c8209527a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.593609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9c26783f-4c77-4416-b9af-a113f324a9d6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.599255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9c26783f-4c77-4416-b9af-a113f324a9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.617220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 768/1544] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.618396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26783f-4c77-4416-b9af-a113f324a9d6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:05.621030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-cd43eb2b-ae6b-48b3-bdef-7c8209527a7f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.622394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cd43eb2b-ae6b-48b3-bdef-7c8209527a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.637101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 777/1545] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:05.641801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43eb2b-ae6b-48b3-bdef-7c8209527a7f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:05.642064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26783f-4c77-4416-b9af-a113f324a9d6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:05.650147 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544c13d-698c-4f85-a05e-974f18cfa246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:05.651084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544c13d-698c-4f85-a05e-974f18cfa246 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.651760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544c13d-698c-4f85-a05e-974f18cfa246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.652547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544c13d-698c-4f85-a05e-974f18cfa246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.653428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544c13d-698c-4f85-a05e-974f18cfa246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:05.655283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-f544c13d-698c-4f85-a05e-974f18cfa246 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:43:05.655283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:43:05.655283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:43:05.657209 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544c13d-698c-4f85-a05e-974f18cfa246 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:05.657837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43eb2b-ae6b-48b3-bdef-7c8209527a7f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:05.674962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e9d49-7a83-465d-969e-fdc1e1b29cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:05.675625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e9d49-7a83-465d-969e-fdc1e1b29cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.675913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e9d49-7a83-465d-969e-fdc1e1b29cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.676190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e9d49-7a83-465d-969e-fdc1e1b29cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.676974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e9d49-7a83-465d-969e-fdc1e1b29cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:05.677179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e9d49-7a83-465d-969e-fdc1e1b29cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:05.678799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e9d49-7a83-465d-969e-fdc1e1b29cfa admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:05.694816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 778/1546] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:43:05.694816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5707d-3845-4b49-8c84-6cd5dc8631ea None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.694816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5707d-3845-4b49-8c84-6cd5dc8631ea None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.694816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5707d-3845-4b49-8c84-6cd5dc8631ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.714570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 769/1547] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/projects => generated 393 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:05.716392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3df5707d-3845-4b49-8c84-6cd5dc8631ea None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.716887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a861e-5d68-4a3d-adf5-d164eaa21bc7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.717361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a861e-5d68-4a3d-adf5-d164eaa21bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.717628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3df5707d-3845-4b49-8c84-6cd5dc8631ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.718041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a861e-5d68-4a3d-adf5-d164eaa21bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.745389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8b7a861e-5d68-4a3d-adf5-d164eaa21bc7 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.747243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b7a861e-5d68-4a3d-adf5-d164eaa21bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.805562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 779/1548] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.805562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e85ef-089d-41a7-b332-bc7866344256 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.805562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e85ef-089d-41a7-b332-bc7866344256 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.805562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e85ef-089d-41a7-b332-bc7866344256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.822459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 770/1549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.822459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d5e8-d052-4e91-b06d-bc389b54fb7c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.822459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d5e8-d052-4e91-b06d-bc389b54fb7c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.822459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d5e8-d052-4e91-b06d-bc389b54fb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.822956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-527e85ef-089d-41a7-b332-bc7866344256 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.824229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-527e85ef-089d-41a7-b332-bc7866344256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.835120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 780/1550] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.836036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e85ef-089d-41a7-b332-bc7866344256 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:05.840795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7b79d5e8-d052-4e91-b06d-bc389b54fb7c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.842462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7b79d5e8-d052-4e91-b06d-bc389b54fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.847457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e85ef-089d-41a7-b332-bc7866344256 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:05.851374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 771/1551] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.852443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cd6bf-0ad2-4a93-b31c-2e2a679d5fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:05.853057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cd6bf-0ad2-4a93-b31c-2e2a679d5fed admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.853364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cd6bf-0ad2-4a93-b31c-2e2a679d5fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.853636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cd6bf-0ad2-4a93-b31c-2e2a679d5fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.855107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cd6bf-0ad2-4a93-b31c-2e2a679d5fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:05.856105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79d5e8-d052-4e91-b06d-bc389b54fb7c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:05.857093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cd6bf-0ad2-4a93-b31c-2e2a679d5fed admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:05.866037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79d5e8-d052-4e91-b06d-bc389b54fb7c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:05.871092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c2d7d-e989-4f41-8a59-a659e963fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:05.871832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2d7d-e989-4f41-8a59-a659e963fab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.872112 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2d7d-e989-4f41-8a59-a659e963fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.872385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2d7d-e989-4f41-8a59-a659e963fab5 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.873015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c2d7d-e989-4f41-8a59-a659e963fab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:05.875051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c2d7d-e989-4f41-8a59-a659e963fab5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:05.889023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 781/1552] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:05.891767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf928c-feea-4b04-8453-f862fa6ec16a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.892153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf928c-feea-4b04-8453-f862fa6ec16a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.892421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf928c-feea-4b04-8453-f862fa6ec16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.899223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 772/1553] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:05.900666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b0c07-0194-401a-8342-f4e7bb145f8f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.902094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b0c07-0194-401a-8342-f4e7bb145f8f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.902386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b0c07-0194-401a-8342-f4e7bb145f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.912578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-eccf928c-feea-4b04-8453-f862fa6ec16a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.913802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-eccf928c-feea-4b04-8453-f862fa6ec16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.921480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f96b0c07-0194-401a-8342-f4e7bb145f8f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:05.922632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f96b0c07-0194-401a-8342-f4e7bb145f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:05.977190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 782/1554] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.980947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf909a-1618-4dd2-b755-6df2485e1834 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:05.981478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf909a-1618-4dd2-b755-6df2485e1834 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:05.981719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf909a-1618-4dd2-b755-6df2485e1834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:05.984247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 773/1555] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:05.985264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b0c07-0194-401a-8342-f4e7bb145f8f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:05.995597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b0c07-0194-401a-8342-f4e7bb145f8f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.000124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27b9cd-8518-498a-a583-5c7dd0a5fb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.000732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27b9cd-8518-498a-a583-5c7dd0a5fb00 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.001011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27b9cd-8518-498a-a583-5c7dd0a5fb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.001280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27b9cd-8518-498a-a583-5c7dd0a5fb00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.001930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27b9cd-8518-498a-a583-5c7dd0a5fb00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:06.002228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27b9cd-8518-498a-a583-5c7dd0a5fb00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.003867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27b9cd-8518-498a-a583-5c7dd0a5fb00 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.007550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-acbf909a-1618-4dd2-b755-6df2485e1834 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:06.008798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-acbf909a-1618-4dd2-b755-6df2485e1834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:06.016143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 774/1556] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:06.017644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a205eda-02c2-465e-b61a-0dcf9661cbdd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.017937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a205eda-02c2-465e-b61a-0dcf9661cbdd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.018216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a205eda-02c2-465e-b61a-0dcf9661cbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.022246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 783/1557] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:06.022246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472548-32f3-49cb-8ca4-1161c0d61a31 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.022246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472548-32f3-49cb-8ca4-1161c0d61a31 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.022246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472548-32f3-49cb-8ca4-1161c0d61a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.038169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6a205eda-02c2-465e-b61a-0dcf9661cbdd None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:06.039444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6a205eda-02c2-465e-b61a-0dcf9661cbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:06.040261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-80472548-32f3-49cb-8ca4-1161c0d61a31 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:06.041414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-80472548-32f3-49cb-8ca4-1161c0d61a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:06.048184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 775/1558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:06.049010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a205eda-02c2-465e-b61a-0dcf9661cbdd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.049456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 784/1559] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:06.050153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472548-32f3-49cb-8ca4-1161c0d61a31 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.059721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472548-32f3-49cb-8ca4-1161c0d61a31 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.062709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a205eda-02c2-465e-b61a-0dcf9661cbdd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.064988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad70a1-8e5c-4328-b3e8-80c06b50698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.065564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad70a1-8e5c-4328-b3e8-80c06b50698f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.065861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad70a1-8e5c-4328-b3e8-80c06b50698f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.066146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad70a1-8e5c-4328-b3e8-80c06b50698f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.066790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad70a1-8e5c-4328-b3e8-80c06b50698f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.067560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fccc44-9dc2-4b6c-ba81-d011e14ff46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.068168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fccc44-9dc2-4b6c-ba81-d011e14ff46e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.068430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad70a1-8e5c-4328-b3e8-80c06b50698f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.068538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fccc44-9dc2-4b6c-ba81-d011e14ff46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.068718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fccc44-9dc2-4b6c-ba81-d011e14ff46e admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.069378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fccc44-9dc2-4b6c-ba81-d011e14ff46e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.071328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fccc44-9dc2-4b6c-ba81-d011e14ff46e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.078527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 785/1560] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:06.079746 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237feb7-1032-492d-b24b-205fbcf41ea5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.080049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237feb7-1032-492d-b24b-205fbcf41ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.080156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:06.080316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237feb7-1032-492d-b24b-205fbcf41ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.099680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 776/1561] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.099680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d6f1b-c1b9-4bbb-9324-e341eafc6b2a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.099680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d6f1b-c1b9-4bbb-9324-e341eafc6b2a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.099680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d6f1b-c1b9-4bbb-9324-e341eafc6b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.104722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4237feb7-1032-492d-b24b-205fbcf41ea5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:06.106065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4237feb7-1032-492d-b24b-205fbcf41ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:06.119242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 786/1562] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:06.119242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237feb7-1032-492d-b24b-205fbcf41ea5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.119889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-003d6f1b-c1b9-4bbb-9324-e341eafc6b2a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:06.121323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-003d6f1b-c1b9-4bbb-9324-e341eafc6b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:06.121466 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:06.127435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237feb7-1032-492d-b24b-205fbcf41ea5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.130404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 777/1563] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:06.131023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d6f1b-c1b9-4bbb-9324-e341eafc6b2a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.132747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.133377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.133748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.134029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.134677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:06.134980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.137123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.140749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d6f1b-c1b9-4bbb-9324-e341eafc6b2a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.145329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.145931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.146214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.146491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.147124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.149575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.149919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 787/1564] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:06.151033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.167926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a245a-9ecc-465e-943d-d5091890f348 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.173664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.174294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.174580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.174858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.175558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:06.175856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.177485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.180326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 778/1565] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.181168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.200309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 788/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:06.200309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.204983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5d54d-5f12-4ecd-be42-13ee06e8b6e8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.260073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aef44e-ed79-417c-91ec-f6d0af08ea47 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.265225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.265873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.266261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.266686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.267379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:06.267741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.267846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.268553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.268716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.268983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.269353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.269662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.271377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.281583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 789/1567] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:06.281737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 779/1568] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:06.282755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.284144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.293183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152327e8-4048-4755-adff-04d9be79a133 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.295507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b280b-3055-4994-9579-a7f68965cdd6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.298189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.298853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.299143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.299486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.300229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.300388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.300982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.301259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.301530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.301836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.302172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:06.302504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.304113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.312394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 790/1569] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:06.316183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.320867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 780/1570] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:06.321905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.332104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610ca84-f0f1-4464-8416-dd72a8d18636 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.333673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649585-812f-4ce8-860d-f0b452d10d48 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.336763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.337346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.337631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.337901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.338632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.339066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.339719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.340006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.340286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.340670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.341010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:06.341320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.342921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.355864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 791/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:06.356662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.368774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ade29f-d75e-4ea6-9635-ef6a4002a78c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.374301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.377814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 781/1572] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.378318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.378318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.378318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.378601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.378692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.378692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:43:06.378692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Aug 24 06:43:06.378692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: Aug 24 06:43:06.378692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.401027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 792/1573] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.402083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.411957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4c83b-882f-4167-a581-5ebbb8711da7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.440678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c0fce-02a6-45c3-b03a-eab1fa0c2756 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.445357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.445975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.446263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.446542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.447258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.449286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.469435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.470138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.470423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.470696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] PATH_INFO: `/v3/projects/2a401374685240dc94fe28ca99ad4e11/users/2c917c21c7974c8d9746e8b05d9c8b9f/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.479878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 782/1574] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.481122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.485849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a401374685240dc94fe28ca99ad4e11, user_id=2c917c21c7974c8d9746e8b05d9c8b9f, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.487092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:06.488590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.502720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 793/1575] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:06 2022] PUT /identity/v3/projects/2a401374685240dc94fe28ca99ad4e11/users/2c917c21c7974c8d9746e8b05d9c8b9f/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:06.503631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.516586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:06.552488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53173f88-35b9-4340-af1c-d62ad34a95ea admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.576182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee00261-371b-45d6-a240-745bec0d85a4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.582663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.583345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.583628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.583908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.584570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.586205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.599373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 794/1576] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:06.600272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.611664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ea161-fe08-4e70-9ba4-349088d10aba admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.616526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.617151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.617272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.617434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.617710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.617817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.618097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.618345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.618454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.618983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.620348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.621116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.648715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 783/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.649304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 795/1578] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.649441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.650126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.712962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d28bc1-7e99-4caf-aaa2-7fe6f14039d0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.717723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.718123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5cda-2841-426e-bfe6-95e0f44b7f3c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.718385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.718671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.718952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.719677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.721722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.722689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.723332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.723620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.723897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.724521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.726584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.743765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 796/1579] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.744894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.755416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064204a-afc5-4fa5-b097-07dd45892f5b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.758585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 784/1580] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.759686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.760394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b33a8-d849-4620-bf9a-3296f35bf0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.761304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b33a8-d849-4620-bf9a-3296f35bf0cd admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.761694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b33a8-d849-4620-bf9a-3296f35bf0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.761980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b33a8-d849-4620-bf9a-3296f35bf0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.762637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b33a8-d849-4620-bf9a-3296f35bf0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.764291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b33a8-d849-4620-bf9a-3296f35bf0cd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.777481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 797/1581] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:06.777481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5326-4c4d-4cf4-a19a-e754d307af47 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.777481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5326-4c4d-4cf4-a19a-e754d307af47 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.777481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa5326-4c4d-4cf4-a19a-e754d307af47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.788061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:06.796668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7eaa5326-4c4d-4cf4-a19a-e754d307af47 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:06.797908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7eaa5326-4c4d-4cf4-a19a-e754d307af47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:06.827728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27252f-77c1-4e69-9578-7e526da54cae admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.832905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.833762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.834044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.835104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.835104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.837163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.861778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 798/1582] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:06.862944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5326-4c4d-4cf4-a19a-e754d307af47 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.864169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 785/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.865828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.877250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa5326-4c4d-4cf4-a19a-e754d307af47 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.877489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc767d-23b0-4837-b977-284663cce64b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.882696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.883164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.883411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.883695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.883802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.883974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] PATH_INFO: `/v3/projects/a41fb206b09941ac80996a3c9cbaae58/users/f264e354671d4456bb33668fb11d9636/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.884129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.884433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.885069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.886605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:06.887627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.903352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41fb206b09941ac80996a3c9cbaae58, user_id=f264e354671d4456bb33668fb11d9636, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.906221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.914765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 786/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:06.915619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.921529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 799/1585] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:06 2022] PUT /identity/v3/projects/a41fb206b09941ac80996a3c9cbaae58/users/f264e354671d4456bb33668fb11d9636/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:06.922686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:06.928131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db63cd-689f-4691-abf8-4cec2db2706c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.984512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0301c-8a55-4b5f-bee7-0d209dfad441 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:06.988401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.989027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.989156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:06.989320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.989602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.989747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:06.990034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:06.990312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:06.990421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.990963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:06.992313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:06.992618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.002474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 800/1586] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.003382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.014391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd8386-a482-4279-bff7-771b06384116 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.014606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 787/1587] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:06 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:07.015287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.022260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.022874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.023159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.023470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.024104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.024729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a6b0c-78ca-469d-b9be-a1375c3f2de5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.026254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.030059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.033627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.033627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.033627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.033627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.033627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.049102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 788/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.049102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.049796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 801/1589] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:07 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:07.050756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.056456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592f89c-1e8a-41a6-aac3-e02ae0b11df3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.062512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.063149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.063462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.063735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.064380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.065973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.076308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 789/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.077478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.088251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133c6ab-dffa-401a-b2f2-e037c84e46ca admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.093235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c04bf-d443-428b-8fc0-3253108a20dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.093828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c04bf-d443-428b-8fc0-3253108a20dd admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.094112 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c04bf-d443-428b-8fc0-3253108a20dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.094388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c04bf-d443-428b-8fc0-3253108a20dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.095022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c04bf-d443-428b-8fc0-3253108a20dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.096684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c04bf-d443-428b-8fc0-3253108a20dd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.108916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 790/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.110164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc33bf-ce51-4540-b9c6-72e509becb86 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.110301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f4bc4-c847-4003-8893-637f6d3b11d7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.110512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc33bf-ce51-4540-b9c6-72e509becb86 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.110717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc33bf-ce51-4540-b9c6-72e509becb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.114213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.114832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.115123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.115440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.126999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.128521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.130664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f5fc33bf-ce51-4540-b9c6-72e509becb86 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:07.131910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f5fc33bf-ce51-4540-b9c6-72e509becb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:07.139219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 802/1592] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:07.139871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.148363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 791/1593] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:07.148363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc33bf-ce51-4540-b9c6-72e509becb86 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.155796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4733339-213c-40c8-91a8-447da9b791f0 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.159048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc33bf-ce51-4540-b9c6-72e509becb86 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.160845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.161481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.161764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.162037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.162687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.164285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.164425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.164901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.165188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.165492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.166230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.168031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.179888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 803/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.179888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.184001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 792/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.184001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.188460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b13b-e557-4e3a-8afa-033ff21bf692 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.192938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf076b3f-dede-4559-ab61-45b83fd314e4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.193465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.194171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.194549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.194681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] PATH_INFO: `/v3/projects/0adb8c7877ba4820b28971463c2241a9/users/2a6086795009437f8e2f7cbd5d902490/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.198522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.198946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.199254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.199535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.200317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.201925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.211826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0adb8c7877ba4820b28971463c2241a9, user_id=2a6086795009437f8e2f7cbd5d902490, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.213527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.223930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 793/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.224901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.232761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 804/1597] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:07 2022] PUT /identity/v3/projects/0adb8c7877ba4820b28971463c2241a9/users/2a6086795009437f8e2f7cbd5d902490/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:07.233983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.235494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865d5f4-939d-4697-91f1-bdca40c7eb94 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.256960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:07.272772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:07.305366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.305899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.306342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.306614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.307401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.309146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.309903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7e5bf-a0fb-40cf-a97c-7a9477d8e175 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.315099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.315938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.316312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.316584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.317380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.319251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.323844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 794/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.323844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.333559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 805/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.336573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.352101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243d7ac-0b98-4830-ba09-3c2fd9fd009a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.359825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.360762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.361559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.361847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] PATH_INFO: `/v3/projects/59b7ab43f7824f6c8a17988db2a25807/users/7f3061468e644588a9f7f8de2af94827/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.385899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b7ab43f7824f6c8a17988db2a25807, user_id=7f3061468e644588a9f7f8de2af94827, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.387496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073677-bb94-4800-967d-9ec6224672e3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.389199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.391300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.391923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.392211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.392491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.402984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.404562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.408975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 806/1600] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:07 2022] PUT /identity/v3/projects/59b7ab43f7824f6c8a17988db2a25807/users/7f3061468e644588a9f7f8de2af94827/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:07.408975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.417117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 795/1601] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:07.418190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.430012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f9cdf-7f6d-4035-8a3f-824ad31f2b1b service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.500472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081319f2-438d-4a36-b2e2-9f98ccb87efb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.505442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.505732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5a390-68b5-4c35-95db-329abcfaafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.506364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5a390-68b5-4c35-95db-329abcfaafd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.506506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.506702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5a390-68b5-4c35-95db-329abcfaafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.506931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.507056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5a390-68b5-4c35-95db-329abcfaafd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.507374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.507810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5a390-68b5-4c35-95db-329abcfaafd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.508793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.509518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5a390-68b5-4c35-95db-329abcfaafd4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.511042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.522607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 807/1602] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.523729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5376-7d73-4bf7-b47a-e3ce1158c346 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.524084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5376-7d73-4bf7-b47a-e3ce1158c346 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.524387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5376-7d73-4bf7-b47a-e3ce1158c346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.533855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 796/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:07.533855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.546134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b26cc-d538-4a45-a18a-f462e57d958a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.552246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.552924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.553204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.553477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] PATH_INFO: `/v3/projects/c484a5dd3cb142b99557d236e1aa4c96/users/7f2b671aef56475e8828dcbd07190aff/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.559556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ae5e5376-7d73-4bf7-b47a-e3ce1158c346 None None] MFA Rules not processed for user `2a6086795009437f8e2f7cbd5d902490`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:07.560918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ae5e5376-7d73-4bf7-b47a-e3ce1158c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:07.579132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c484a5dd3cb142b99557d236e1aa4c96, user_id=7f2b671aef56475e8828dcbd07190aff, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.580149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:43:07.580149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:43:07.580149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:43:07.582294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.603390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 797/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:07 2022] PUT /identity/v3/projects/c484a5dd3cb142b99557d236e1aa4c96/users/7f2b671aef56475e8828dcbd07190aff/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:07.605030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.660870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 808/1605] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:07 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:07.662228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633af46a-0c4e-43ee-a54b-912db62b2b33 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.662574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633af46a-0c4e-43ee-a54b-912db62b2b33 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.662857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633af46a-0c4e-43ee-a54b-912db62b2b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.694085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:07.694741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-633af46a-0c4e-43ee-a54b-912db62b2b33 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:07.698793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-633af46a-0c4e-43ee-a54b-912db62b2b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:07.740305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cbef5-c937-42e0-af28-7c27b30a5797 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.747102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.747874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.748161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.748438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] PATH_INFO: `/v3/projects/5b4541156c1f4efab3a61dd6935638bb/users/1d26cf886087408097cc2e435a785454/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.753356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:07.775768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4541156c1f4efab3a61dd6935638bb, user_id=1d26cf886087408097cc2e435a785454, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.779673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.811849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 798/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:07 2022] PUT /identity/v3/projects/5b4541156c1f4efab3a61dd6935638bb/users/1d26cf886087408097cc2e435a785454/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:07.811849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.816904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 809/1607] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:07.818160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633af46a-0c4e-43ee-a54b-912db62b2b33 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.841870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633af46a-0c4e-43ee-a54b-912db62b2b33 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.892502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a43db-d1cd-4932-b9d0-964dbbbf2f32 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:07.898371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d8d5f-cc81-4a32-b55c-10c0a9e825db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.899449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8d5f-cc81-4a32-b55c-10c0a9e825db admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.899449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8d5f-cc81-4a32-b55c-10c0a9e825db admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.899690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8d5f-cc81-4a32-b55c-10c0a9e825db admin admin] PATH_INFO: `/v3/projects/16c7f1d71bf24c558c566f71aa177447/users/1f345e623d524d868cfa3115ff5411bf/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.923980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d8d5f-cc81-4a32-b55c-10c0a9e825db admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c7f1d71bf24c558c566f71aa177447, user_id=1f345e623d524d868cfa3115ff5411bf, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.927958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d8d5f-cc81-4a32-b55c-10c0a9e825db admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.931057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:07.931807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.932915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.933301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] PATH_INFO: `/v3/projects/0aec98515a95467aa9a43853bfdcb7d7/users/bfb5dc1fa16f465ba1ebca000b665d46/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.948518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 799/1608] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:07 2022] PUT /identity/v3/projects/16c7f1d71bf24c558c566f71aa177447/users/1f345e623d524d868cfa3115ff5411bf/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:07.950037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad29b-08ce-4f2e-9400-e9c603bec854 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:07.950332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad29b-08ce-4f2e-9400-e9c603bec854 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:07.950661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad29b-08ce-4f2e-9400-e9c603bec854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:07.962955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aec98515a95467aa9a43853bfdcb7d7, user_id=bfb5dc1fa16f465ba1ebca000b665d46, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:07.965889 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:07.982697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 810/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:07 2022] PUT /identity/v3/projects/0aec98515a95467aa9a43853bfdcb7d7/users/bfb5dc1fa16f465ba1ebca000b665d46/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:07.983334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:07.995386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-278ad29b-08ce-4f2e-9400-e9c603bec854 None None] MFA Rules not processed for user `7f2b671aef56475e8828dcbd07190aff`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:07.996708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-278ad29b-08ce-4f2e-9400-e9c603bec854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:08.050421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17c4a1-92d0-489d-96e0-60d6499c7e02 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.055452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.056323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.056708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.057091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.058000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.060305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.071540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 800/1610] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:08.078567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88309d-9a89-4284-9dd5-2ca0c4773cc6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.078891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88309d-9a89-4284-9dd5-2ca0c4773cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.079178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88309d-9a89-4284-9dd5-2ca0c4773cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.095031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 811/1611] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:07 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:08.096070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.100579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ad88309d-9a89-4284-9dd5-2ca0c4773cc6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:08.101793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ad88309d-9a89-4284-9dd5-2ca0c4773cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:08.128895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:08.152479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:08.173619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cc339-c0c8-4a5a-b65e-0f0f8fae5e28 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.177393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b394661-e84c-4678-ae15-398afd2187f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.178107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394661-e84c-4678-ae15-398afd2187f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.178458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394661-e84c-4678-ae15-398afd2187f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.178816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394661-e84c-4678-ae15-398afd2187f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.179513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 801/1612] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:08.182878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeddb12-feba-4b7d-a01a-e8320d91fd1f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.182878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeddb12-feba-4b7d-a01a-e8320d91fd1f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.182878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeddb12-feba-4b7d-a01a-e8320d91fd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.194606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b394661-e84c-4678-ae15-398afd2187f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.196270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b394661-e84c-4678-ae15-398afd2187f1 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.209525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 812/1613] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:08.210720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57228a-9cd0-43b7-a0c7-fb5e3bf1c98e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.211146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57228a-9cd0-43b7-a0c7-fb5e3bf1c98e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.211585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57228a-9cd0-43b7-a0c7-fb5e3bf1c98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.222047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dfeddb12-feba-4b7d-a01a-e8320d91fd1f None None] MFA Rules not processed for user `1f345e623d524d868cfa3115ff5411bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:08.223442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dfeddb12-feba-4b7d-a01a-e8320d91fd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:08.249729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4e57228a-9cd0-43b7-a0c7-fb5e3bf1c98e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:08.276778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4e57228a-9cd0-43b7-a0c7-fb5e3bf1c98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:08.299023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 813/1614] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:08.300215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57228a-9cd0-43b7-a0c7-fb5e3bf1c98e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.337775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 802/1615] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:08.340640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeddb12-feba-4b7d-a01a-e8320d91fd1f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.377456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeddb12-feba-4b7d-a01a-e8320d91fd1f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.391144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.392409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.392810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.393195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.393967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.402895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.455807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 803/1616] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:08 2022] POST /identity/v3/users => generated 309 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:08.455807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.456085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57228a-9cd0-43b7-a0c7-fb5e3bf1c98e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.460377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.461922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.462389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.462816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.477268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cc013-a175-402c-aa6e-43ae7faa90cd admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.484659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.485350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.485691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.485978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.498807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.500441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.519101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 804/1617] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:08.520214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.531236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109523ec-b90e-46b2-9d7b-c45b41d2a112 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.536295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cd483-839d-4739-9667-8b101436b618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.536847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cd483-839d-4739-9667-8b101436b618 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.537127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cd483-839d-4739-9667-8b101436b618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.537411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cd483-839d-4739-9667-8b101436b618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.540638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.545334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.552080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cd483-839d-4739-9667-8b101436b618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.553822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cd483-839d-4739-9667-8b101436b618 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.569214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 805/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:08.571070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8207de-e24a-4c69-b2e8-9b47705d3f74 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.571470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8207de-e24a-4c69-b2e8-9b47705d3f74 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.571749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 814/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:08.571869 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8207de-e24a-4c69-b2e8-9b47705d3f74 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.573978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 806/1620] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 06:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:08.579814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5ebf-82d4-4388-93cf-108d4f83adfd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.580445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5ebf-82d4-4388-93cf-108d4f83adfd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.581210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.581440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5ebf-82d4-4388-93cf-108d4f83adfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.612885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70e090-9ad3-4e05-9a0e-495c36d289fb service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.627880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.628584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.628868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.629166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.630062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.633895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.646816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-18ab5ebf-82d4-4388-93cf-108d4f83adfd None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:08.648622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-18ab5ebf-82d4-4388-93cf-108d4f83adfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:08.660381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 815/1621] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:08.669687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.689871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e2682-9f72-4e7f-bbc5-4bf0f5c683ee admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.704240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.705254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.705424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.707772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] PATH_INFO: `/v3/projects/0b21099d59c744d9acf47b2b6ee8c3cb/users/395cc90254db48b7ac4b36973c20a89f/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.726053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:08.745819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b21099d59c744d9acf47b2b6ee8c3cb, user_id=395cc90254db48b7ac4b36973c20a89f, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.748646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.757216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 807/1622] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 06:43:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:08.766503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5ebf-82d4-4388-93cf-108d4f83adfd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.780730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 816/1623] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:08 2022] PUT /identity/v3/projects/0b21099d59c744d9acf47b2b6ee8c3cb/users/395cc90254db48b7ac4b36973c20a89f/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:08.782465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.797406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5ebf-82d4-4388-93cf-108d4f83adfd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.875160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:08.890995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f9420-6137-4dd1-82e9-a8d3922f9a56 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.900873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.903054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.903572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.904351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.905247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.907671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:08.921986 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 817/1624] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:08.926024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:08.957103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf5034-5c91-46f7-ab14-6e0f9153c357 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:08.968316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.968972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.969259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.969570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.977548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c238e-60e7-4519-9a16-bdb93bb7e6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:08.978531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c238e-60e7-4519-9a16-bdb93bb7e6f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:08.978676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c238e-60e7-4519-9a16-bdb93bb7e6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:08.978891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c238e-60e7-4519-9a16-bdb93bb7e6f5 admin admin] PATH_INFO: `/v3/projects/0b21099d59c744d9acf47b2b6ee8c3cb/users/395cc90254db48b7ac4b36973c20a89f/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:08.991079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c238e-60e7-4519-9a16-bdb93bb7e6f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b21099d59c744d9acf47b2b6ee8c3cb, user_id=395cc90254db48b7ac4b36973c20a89f, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:08.997624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c238e-60e7-4519-9a16-bdb93bb7e6f5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:09.035476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 818/1625] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:08 2022] PUT /identity/v3/projects/0b21099d59c744d9acf47b2b6ee8c3cb/users/395cc90254db48b7ac4b36973c20a89f/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:09.045787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3db5-5bcf-4d32-a4e8-5e6093cdd3ef None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:09.046200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3db5-5bcf-4d32-a4e8-5e6093cdd3ef None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:09.046638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3db5-5bcf-4d32-a4e8-5e6093cdd3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:09.144106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-438d3db5-5bcf-4d32-a4e8-5e6093cdd3ef None None] MFA Rules not processed for user `395cc90254db48b7ac4b36973c20a89f`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:09.151678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-438d3db5-5bcf-4d32-a4e8-5e6093cdd3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:09.160739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:09.163135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:09.204057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 808/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:09.320655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 819/1627] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:09 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:09.321015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:09.440328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50324530-19b8-4b47-b311-0c8cf66f3878 service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:09.534442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:09.548005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c18f20-882a-47b2-ac87-5b6bee60ca7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:09.549737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c18f20-882a-47b2-ac87-5b6bee60ca7c service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:09.550302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c18f20-882a-47b2-ac87-5b6bee60ca7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:09.550779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c18f20-882a-47b2-ac87-5b6bee60ca7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:09.574335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c18f20-882a-47b2-ac87-5b6bee60ca7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:09.576068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c18f20-882a-47b2-ac87-5b6bee60ca7c service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:09.600783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8530-b749-428f-8160-90a027022175 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:09.601508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 809/1628] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:09 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:09.606698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8530-b749-428f-8160-90a027022175 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:09.607394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f8530-b749-428f-8160-90a027022175 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:09.612069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 820/1629] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 06:43:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:43:09.619356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c7628-49c8-45cd-b9d9-84ab8aeef37e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:09.619532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c7628-49c8-45cd-b9d9-84ab8aeef37e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:09.620036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c7628-49c8-45cd-b9d9-84ab8aeef37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:09.671455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3d6c7628-49c8-45cd-b9d9-84ab8aeef37e None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:09.672863 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3d6c7628-49c8-45cd-b9d9-84ab8aeef37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:09.840853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 810/1630] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:09.848045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8530-b749-428f-8160-90a027022175 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:09.866392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f8530-b749-428f-8160-90a027022175 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:09.876180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcef2e-5ca0-4996-a9b3-cc68fb55dbf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:09.878435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcef2e-5ca0-4996-a9b3-cc68fb55dbf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:09.879751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcef2e-5ca0-4996-a9b3-cc68fb55dbf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:09.880879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcef2e-5ca0-4996-a9b3-cc68fb55dbf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:09.912068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcef2e-5ca0-4996-a9b3-cc68fb55dbf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:09.916270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcef2e-5ca0-4996-a9b3-cc68fb55dbf2 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:09.959575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 821/1631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:10.150166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c7628-49c8-45cd-b9d9-84ab8aeef37e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:10.276783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c7628-49c8-45cd-b9d9-84ab8aeef37e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:10.295580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:10.296246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:10.296533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:10.296814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] PATH_INFO: `/v3/users/2a6086795009437f8e2f7cbd5d902490` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:10.297976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6086795009437f8e2f7cbd5d902490)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:10.299713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:43:10.299713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 24 06:43:10.299713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: Aug 24 06:43:10.301232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:10.408497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6086795009437f8e2f7cbd5d902490'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:10.438331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6086795009437f8e2f7cbd5d902490'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:10.449870 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6086795009437f8e2f7cbd5d902490'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:10.450970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6086795009437f8e2f7cbd5d902490'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:10.451942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6086795009437f8e2f7cbd5d902490'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:10.487736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:10.500579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8d3d3-754f-4dc1-b9b1-305998f98441 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:10.500878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 811/1632] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:10 2022] DELETE /identity/v3/users/2a6086795009437f8e2f7cbd5d902490 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:10.501395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8d3d3-754f-4dc1-b9b1-305998f98441 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:10.501683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8d3d3-754f-4dc1-b9b1-305998f98441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:10.537541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-81c8d3d3-754f-4dc1-b9b1-305998f98441 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:10.539662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-81c8d3d3-754f-4dc1-b9b1-305998f98441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:10.754098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 822/1633] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:10.847902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:10.849736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:11.038295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66415f84-696b-4667-ae06-67df07facf4c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:11.046936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5167b8-086d-4025-b94b-e0a9a7752a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:11.047605 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5167b8-086d-4025-b94b-e0a9a7752a60 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:11.047889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5167b8-086d-4025-b94b-e0a9a7752a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:11.048162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5167b8-086d-4025-b94b-e0a9a7752a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:11.067102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5167b8-086d-4025-b94b-e0a9a7752a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:11.071476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5167b8-086d-4025-b94b-e0a9a7752a60 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:11.172266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 812/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:11.399891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ba1b-c8f5-487c-97a9-43ed6497edd9 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:11.400104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ba1b-c8f5-487c-97a9-43ed6497edd9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:11.400383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ba1b-c8f5-487c-97a9-43ed6497edd9 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:11.401830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 823/1635] 10.222.0.52 () {54 vars in 962 bytes} [Wed Aug 24 06:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:11.418076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e0e52-858d-4905-b0a2-d2c2cb05d2ab None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:11.418368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e0e52-858d-4905-b0a2-d2c2cb05d2ab None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:11.418644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e0e52-858d-4905-b0a2-d2c2cb05d2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:11.474387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-654e0e52-858d-4905-b0a2-d2c2cb05d2ab None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:11.488178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-654e0e52-858d-4905-b0a2-d2c2cb05d2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:11.613598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 813/1636] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Aug 24 06:43:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:11.627333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0b48-7de6-4dce-8e94-67108115e9ff None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:11.627618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0b48-7de6-4dce-8e94-67108115e9ff None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:11.627893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0b48-7de6-4dce-8e94-67108115e9ff None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:11.629520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 824/1637] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 06:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:11.636209 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:11.636482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:11.636771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:11.659957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:11.661791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:11.673753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 814/1638] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:11.689657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f0b48-7de6-4dce-8e94-67108115e9ff None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:11.692319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a02f0b48-7de6-4dce-8e94-67108115e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:11.769698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:11.784881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f0b48-7de6-4dce-8e94-67108115e9ff None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:11.792018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b66cdc-8efb-44fa-802d-0a2f82b504a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:11.794190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66cdc-8efb-44fa-802d-0a2f82b504a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:11.794727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66cdc-8efb-44fa-802d-0a2f82b504a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:11.795122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66cdc-8efb-44fa-802d-0a2f82b504a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:11.813989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b66cdc-8efb-44fa-802d-0a2f82b504a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:11.816596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b66cdc-8efb-44fa-802d-0a2f82b504a8 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:11.841297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 825/1639] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:13.927264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:13.929937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:13.962476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363735c-e37b-43a0-b924-9d2fe5869820 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:13.982171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:13.983988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:13.986462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:13.986761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] PATH_INFO: `/v3/users/1f345e623d524d868cfa3115ff5411bf` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:13.994395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f345e623d524d868cfa3115ff5411bf)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:14.002931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:14.130329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f345e623d524d868cfa3115ff5411bf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.144202 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f345e623d524d868cfa3115ff5411bf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.167683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f345e623d524d868cfa3115ff5411bf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.168583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f345e623d524d868cfa3115ff5411bf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.169217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f345e623d524d868cfa3115ff5411bf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.194203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:14.200886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 815/1640] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:13 2022] DELETE /identity/v3/users/1f345e623d524d868cfa3115ff5411bf => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:14.207034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:14.207758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:14.207758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:14.253108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:14.254726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:14.345351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 826/1641] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:14.399017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:14.401724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:14.514989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:14.516618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:14.534358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47705ee-1fc5-45af-b9e0-6814652cfea1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:14.539360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf3746-1c1e-49b3-a4e0-02222d5a55b1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:14.549669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:14.550583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:14.551025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:14.551475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:14.554114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:14.555142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:14.555306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:14.555658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] PATH_INFO: `/v3/projects/0adb8c7877ba4820b28971463c2241a9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:14.559671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0adb8c7877ba4820b28971463c2241a9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:14.560906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:43:14.560906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Aug 24 06:43:14.560906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: Aug 24 06:43:14.566275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:14.577380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:14.581094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:14.604827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 816/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:14.749285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0adb8c7877ba4820b28971463c2241a9', 'request_id': 'req-e1481ff1-e483-424f-afbc-d330dcc99645'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.832397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0adb8c7877ba4820b28971463c2241a9', 'request_id': 'req-e1481ff1-e483-424f-afbc-d330dcc99645'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.844595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0adb8c7877ba4820b28971463c2241a9', 'request_id': 'req-e1481ff1-e483-424f-afbc-d330dcc99645'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.874036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] The token cache is being invalidate because project 0adb8c7877ba4820b28971463c2241a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:14.874624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.875504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:14.877987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 827/1643] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:14 2022] DELETE /identity/v3/projects/0adb8c7877ba4820b28971463c2241a9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:14.882086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:14.897578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664dde5-f7f9-4b30-8c57-0ec825f51128 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:15.050811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:15.053501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:15.054038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:15.054525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] PATH_INFO: `/v3/users/395cc90254db48b7ac4b36973c20a89f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:15.061228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=395cc90254db48b7ac4b36973c20a89f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:15.064113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:15.150917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395cc90254db48b7ac4b36973c20a89f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:15.164473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395cc90254db48b7ac4b36973c20a89f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:15.185544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395cc90254db48b7ac4b36973c20a89f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:15.186576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395cc90254db48b7ac4b36973c20a89f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:15.187922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395cc90254db48b7ac4b36973c20a89f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:15.252116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 817/1644] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:14 2022] DELETE /identity/v3/users/395cc90254db48b7ac4b36973c20a89f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:16.117970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:16.123907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:16.175556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481ff1-e483-424f-afbc-d330dcc99645 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:16.332584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:16.334703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:16.409014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:16.409973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:16.410413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:16.410836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] PATH_INFO: `/v3/projects/16c7f1d71bf24c558c566f71aa177447` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:16.412751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c7f1d71bf24c558c566f71aa177447)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:16.414896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:16.448942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:16.461391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:16.524481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c7f1d71bf24c558c566f71aa177447', 'request_id': 'req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:16.537356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c7f1d71bf24c558c566f71aa177447', 'request_id': 'req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:16.553991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e231e-0e90-4764-bdc5-9bc54207b85e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:16.556832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c7f1d71bf24c558c566f71aa177447', 'request_id': 'req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:16.611596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] The token cache is being invalidate because project 16c7f1d71bf24c558c566f71aa177447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:16.612319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:16.614893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f20b6c5c-0732-43c9-9f41-b55f28c1cd2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:16.620694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 828/1645] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:16 2022] DELETE /identity/v3/projects/16c7f1d71bf24c558c566f71aa177447 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:16.626380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90683c86-f308-4995-9c2e-015adb342a25 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:16.628677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90683c86-f308-4995-9c2e-015adb342a25 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:16.632044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90683c86-f308-4995-9c2e-015adb342a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:16.700349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-90683c86-f308-4995-9c2e-015adb342a25 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:16.701629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-90683c86-f308-4995-9c2e-015adb342a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:16.838860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:16.840875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:16.841262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:16.842546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] PATH_INFO: `/v3/projects/0b21099d59c744d9acf47b2b6ee8c3cb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:16.846545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 829/1646] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:16.852454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b21099d59c744d9acf47b2b6ee8c3cb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:16.853730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243334d3-4472-4356-9c6c-6c0964334832 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:16.854181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243334d3-4472-4356-9c6c-6c0964334832 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:16.854506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243334d3-4472-4356-9c6c-6c0964334832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:16.866494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:16.895374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-243334d3-4472-4356-9c6c-6c0964334832 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:16.896672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-243334d3-4472-4356-9c6c-6c0964334832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:16.930973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 830/1647] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:16.936477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243334d3-4472-4356-9c6c-6c0964334832 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:16.960412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243334d3-4472-4356-9c6c-6c0964334832 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:16.975255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b21099d59c744d9acf47b2b6ee8c3cb', 'request_id': 'req-1338b65e-3dd5-47c5-96f3-67a0088463f8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:16.984264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b21099d59c744d9acf47b2b6ee8c3cb', 'request_id': 'req-1338b65e-3dd5-47c5-96f3-67a0088463f8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:16.993124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b21099d59c744d9acf47b2b6ee8c3cb', 'request_id': 'req-1338b65e-3dd5-47c5-96f3-67a0088463f8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:17.012662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] The token cache is being invalidate because project 0b21099d59c744d9acf47b2b6ee8c3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:17.013117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:17.013888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1338b65e-3dd5-47c5-96f3-67a0088463f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:17.021054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 818/1648] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:16 2022] DELETE /identity/v3/projects/0b21099d59c744d9acf47b2b6ee8c3cb => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:17.023616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.023903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.024152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.062666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:17.064161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:17.139283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29515fa-7c99-4aba-aa6e-a7a9ec47c511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:17.140461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29515fa-7c99-4aba-aa6e-a7a9ec47c511 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.140942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29515fa-7c99-4aba-aa6e-a7a9ec47c511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.141400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29515fa-7c99-4aba-aa6e-a7a9ec47c511 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.142251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29515fa-7c99-4aba-aa6e-a7a9ec47c511 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:17.142735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29515fa-7c99-4aba-aa6e-a7a9ec47c511 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:17.152115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29515fa-7c99-4aba-aa6e-a7a9ec47c511 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:17.214972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 831/1649] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:43:17.216556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d421-87d5-46f8-84fe-4cd673071bd4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.216852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d421-87d5-46f8-84fe-4cd673071bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.217133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d421-87d5-46f8-84fe-4cd673071bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.251743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:17.270052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e586d421-87d5-46f8-84fe-4cd673071bd4 None None] MFA Rules not processed for user `f264e354671d4456bb33668fb11d9636`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:17.271617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e586d421-87d5-46f8-84fe-4cd673071bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:17.290850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 819/1650] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:17.296256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:17.297641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:17.298672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:17.322034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb36bd5-6fbc-424e-8b1e-48a7959a0920 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:17.337782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85a192-619b-4985-a5c1-13c233f20081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:17.338601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85a192-619b-4985-a5c1-13c233f20081 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.339070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85a192-619b-4985-a5c1-13c233f20081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.339415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85a192-619b-4985-a5c1-13c233f20081 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.340489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85a192-619b-4985-a5c1-13c233f20081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:17.342335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85a192-619b-4985-a5c1-13c233f20081 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:17.418581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 820/1651] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:17 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:17.418581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.418943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.419422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.427342 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 832/1652] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:17.436059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d421-87d5-46f8-84fe-4cd673071bd4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:17.456614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d421-87d5-46f8-84fe-4cd673071bd4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:17.469405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:17.470871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:17.555784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:17.557410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.557701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.557988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.558653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:17.561026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:17.604538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 821/1653] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:17.605455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:17.609468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:17.623165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 833/1654] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:17 2022] POST /identity/v3/users => generated 302 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:17.624292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:17.636597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a594163-f146-4084-a12e-abb298aae2ff admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:17.644985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:17.645972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.645972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.646169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.646823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:17.647133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:17.649426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:17.668616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 834/1655] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:17.669656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:17.680625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfa87b-cd24-42e0-96bb-9d0c01960d7b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:17.698065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:17.698479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.698768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.699052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.700693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:17.702395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:17.718317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 835/1656] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:17.719559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:17.745074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4debee-8c2e-4af7-b62c-41f643dd3241 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:17.764053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:17.764452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.764739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.765213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:17.766184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:17.768168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:17.848451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd49c4e-f290-427d-a5a8-9ad0682d55f8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:17.872189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 836/1657] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:17 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:17.873166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:17.889098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:17.889754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:17.890040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:17.890314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.088717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:18.093748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:18.143002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 822/1658] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:18.144287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:18.174787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddff37-ae05-4218-892f-a2efd43cedf9 service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:18.183232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:18.184372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.184825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.185238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.186329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:18.189660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:18.201311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e86b8-8173-4c71-bc57-8f723bdfe186 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:18.225340 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:18.226264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.227314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.227837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.229517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:18.231463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:18.244087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:18.267699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 837/1659] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:18.270859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:18.277449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 823/1660] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:18 2022] POST /identity/v3/users => generated 283 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:18.278754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:18.281727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:18.298741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e1b5-962a-4b1b-9a65-68496af35305 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:18.305842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932f3fd-8e75-4d33-b515-7ff9098df987 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:18.310176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048094c-7b9c-4f0c-8919-c52f41962fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:18.311446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048094c-7b9c-4f0c-8919-c52f41962fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.312196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048094c-7b9c-4f0c-8919-c52f41962fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.312579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048094c-7b9c-4f0c-8919-c52f41962fc6 admin admin] PATH_INFO: `/v3/projects/c7c4e6c97e5842f29be38e8de496b5e7/users/182820d5c8f34678936cfcd354b5115a/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.323528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:18.324285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.324572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.324856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.325526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:18.330283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:18.344069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048094c-7b9c-4f0c-8919-c52f41962fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c4e6c97e5842f29be38e8de496b5e7, user_id=182820d5c8f34678936cfcd354b5115a, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:18.360866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048094c-7b9c-4f0c-8919-c52f41962fc6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:18.368224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 824/1661] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:18.372707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:18.394104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 838/1662] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:18 2022] PUT /identity/v3/projects/c7c4e6c97e5842f29be38e8de496b5e7/users/182820d5c8f34678936cfcd354b5115a/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:18.397111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4104cc-4269-4a16-ac9d-c14000c4f97a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:18.406367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.406798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.407214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.471273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:18.473493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:18.591306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:18.592273 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.592582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.592812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.593557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:18.596502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:18.615248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 825/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:43:18.618846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:18.632989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bad0be-e21e-4ffe-815f-7848bebe6478 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:18.634861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 839/1664] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:18.640065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56cf9d-b9bb-4781-9593-00a998bf6abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:18.640764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56cf9d-b9bb-4781-9593-00a998bf6abc admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.641048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56cf9d-b9bb-4781-9593-00a998bf6abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.641329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56cf9d-b9bb-4781-9593-00a998bf6abc admin admin] PATH_INFO: `/v3/projects/4f5059a58c6d4674a160593e81a69494/users/64d4cd6224da4c4ea4314af5d2b56482/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.662294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56cf9d-b9bb-4781-9593-00a998bf6abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5059a58c6d4674a160593e81a69494, user_id=64d4cd6224da4c4ea4314af5d2b56482, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:18.665555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56cf9d-b9bb-4781-9593-00a998bf6abc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:18.688001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:18.691677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:18.721127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 826/1665] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:18 2022] PUT /identity/v3/projects/4f5059a58c6d4674a160593e81a69494/users/64d4cd6224da4c4ea4314af5d2b56482/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:18.745981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deda5a-281f-4623-b0c9-70d14875f265 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:18.748076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deda5a-281f-4623-b0c9-70d14875f265 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:18.749003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deda5a-281f-4623-b0c9-70d14875f265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:18.915151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-52deda5a-281f-4623-b0c9-70d14875f265 None None] MFA Rules not processed for user `64d4cd6224da4c4ea4314af5d2b56482`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:18.930191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-52deda5a-281f-4623-b0c9-70d14875f265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:19.013884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b240f5-2d69-4eae-a689-6759ab9e5bb3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:19.125492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 827/1666] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:19.127209 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52deda5a-281f-4623-b0c9-70d14875f265 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:19.174512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:19.176375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:19.177701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:19.177990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:19.301573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:19.314634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:19.359625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:19.361142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:19.384426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52deda5a-281f-4623-b0c9-70d14875f265 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:19.398812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 840/1667] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:19.414556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:19.415507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:19.416094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:19.416679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:19.417477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:19.419630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:19.484429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:19.490223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 828/1668] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:19 2022] POST /identity/v3/projects => generated 365 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:19.500044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80518da-c244-4385-8c69-d8358f458a83 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:19.648773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:19.650130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:19.650429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:19.650774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:19.654792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:19.656862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:19.727439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 841/1669] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:19 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:19.732747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:19.747524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b52da0-d4b2-4ffa-bd9a-5f0001b14d7c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:19.758290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:19.763701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:19.764404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:19.764884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:19.768738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:19.773117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:19.802077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 829/1670] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:19.806229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:19.853524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10a784-52f3-45e8-a5b1-534bd406bbd0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:19.868419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:19.869570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:19.869973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:19.871400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] PATH_INFO: `/v3/projects/ec6201927fdb4ea29caad607e7d51da8/users/bdb38d96342b4c789b60fb0e320f70cb/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:19.914663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6201927fdb4ea29caad607e7d51da8, user_id=bdb38d96342b4c789b60fb0e320f70cb, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:19.922074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:19.957188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 842/1671] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:19 2022] PUT /identity/v3/projects/ec6201927fdb4ea29caad607e7d51da8/users/bdb38d96342b4c789b60fb0e320f70cb/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:19.961109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:19.972777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750db36-3806-4bd5-833b-9c63a01e0659 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:20.133576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062893a-6018-41ae-9d5a-d54ac4c61826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:20.135800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062893a-6018-41ae-9d5a-d54ac4c61826 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:20.138086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062893a-6018-41ae-9d5a-d54ac4c61826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:20.138865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062893a-6018-41ae-9d5a-d54ac4c61826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:20.141017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062893a-6018-41ae-9d5a-d54ac4c61826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:20.142703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062893a-6018-41ae-9d5a-d54ac4c61826 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:20.173905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:20.180455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 830/1672] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:43:20.189677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc00c2-051e-414f-846a-496dd9430b4b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:20.199863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:20.203896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:20.204534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:20.205037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] PATH_INFO: `/v3/projects/ec6201927fdb4ea29caad607e7d51da8/users/bdb38d96342b4c789b60fb0e320f70cb/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:20.211664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6201927fdb4ea29caad607e7d51da8, user_id=bdb38d96342b4c789b60fb0e320f70cb, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:20.216526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:20.255241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 843/1673] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:20 2022] PUT /identity/v3/projects/ec6201927fdb4ea29caad607e7d51da8/users/bdb38d96342b4c789b60fb0e320f70cb/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:20.265368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932b2a7-857a-4dff-9c0b-0c6cb05fe964 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:20.265664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932b2a7-857a-4dff-9c0b-0c6cb05fe964 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:20.265948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932b2a7-857a-4dff-9c0b-0c6cb05fe964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:20.322103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c932b2a7-857a-4dff-9c0b-0c6cb05fe964 None None] MFA Rules not processed for user `bdb38d96342b4c789b60fb0e320f70cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:20.326262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c932b2a7-857a-4dff-9c0b-0c6cb05fe964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:20.547101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 831/1674] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:20.558584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:20.563269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:20.799760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc0ce9-c96a-412f-a120-31efa60ec339 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:20.814408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccee4c-146e-4335-b90f-4541e8d7107d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:20.815141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccee4c-146e-4335-b90f-4541e8d7107d service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:20.815520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccee4c-146e-4335-b90f-4541e8d7107d service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:20.815873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccee4c-146e-4335-b90f-4541e8d7107d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:20.836617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccee4c-146e-4335-b90f-4541e8d7107d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:20.838377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccee4c-146e-4335-b90f-4541e8d7107d service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:20.862523 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 844/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:21.932511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932b2a7-857a-4dff-9c0b-0c6cb05fe964 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:22.053865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:22.094998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932b2a7-857a-4dff-9c0b-0c6cb05fe964 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:22.117160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:22.117843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:22.118130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:22.118409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] PATH_INFO: `/v3/users/64d4cd6224da4c4ea4314af5d2b56482` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:22.120010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d4cd6224da4c4ea4314af5d2b56482)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:22.121983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:22.257597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d4cd6224da4c4ea4314af5d2b56482'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.281341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d4cd6224da4c4ea4314af5d2b56482'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.300475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d4cd6224da4c4ea4314af5d2b56482'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.301915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d4cd6224da4c4ea4314af5d2b56482'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.304091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d4cd6224da4c4ea4314af5d2b56482'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.349826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:22.350387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:22.350387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:22.353198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 832/1676] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:21 2022] DELETE /identity/v3/users/64d4cd6224da4c4ea4314af5d2b56482 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:22.385736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:22.387000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:22.477748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:22.596425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 845/1677] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:22.664775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:22.666730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:22.691572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f71b9-46e7-4ba3-971c-33ff9cdacd92 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:22.703447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:22.704360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:22.704652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:22.704933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] PATH_INFO: `/v3/users/7f2b671aef56475e8828dcbd07190aff` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:22.706479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2b671aef56475e8828dcbd07190aff)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:22.708636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:22.756874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:22.759817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:22.812780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2b671aef56475e8828dcbd07190aff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.824688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2b671aef56475e8828dcbd07190aff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.846092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2b671aef56475e8828dcbd07190aff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.846805 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2b671aef56475e8828dcbd07190aff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.847875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fa4c652f-ca04-4a83-85ce-9ecc9ffc6ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2b671aef56475e8828dcbd07190aff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:22.880162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 833/1678] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:22 2022] DELETE /identity/v3/users/7f2b671aef56475e8828dcbd07190aff => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:22.884549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:22.885132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:22.886234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:22.929572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:22.932882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4235ac-6e87-41ba-9dc5-3871c0e735ba None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:22.934436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:23.004698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:23.066153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec2799-8937-4712-8a05-e305181c209e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:23.066835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2799-8937-4712-8a05-e305181c209e service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:23.067823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2799-8937-4712-8a05-e305181c209e service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:23.068194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec2799-8937-4712-8a05-e305181c209e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:23.068406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 834/1679] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:23.074091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fcec2799-8937-4712-8a05-e305181c209e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:23.102028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec2799-8937-4712-8a05-e305181c209e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:23.103500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec2799-8937-4712-8a05-e305181c209e service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:23.128838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 846/1680] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:23.226403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:23.227886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:23.259784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764262eb-e428-4628-a8d5-b5f9ac471086 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:23.275919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42ed8-e018-4f3d-9f51-f90ca27ccfd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:23.277742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42ed8-e018-4f3d-9f51-f90ca27ccfd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:23.278111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42ed8-e018-4f3d-9f51-f90ca27ccfd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:23.278459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42ed8-e018-4f3d-9f51-f90ca27ccfd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:23.296885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42ed8-e018-4f3d-9f51-f90ca27ccfd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:23.298562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42ed8-e018-4f3d-9f51-f90ca27ccfd1 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:23.324934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 835/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:23.930392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:23.935425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:23.935425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:24.002996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] MFA Rules not processed for user `2c917c21c7974c8d9746e8b05d9c8b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:24.004292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:24.207751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeba9c2-b01b-40df-8ee6-5a6af502ab2d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:24.208326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeba9c2-b01b-40df-8ee6-5a6af502ab2d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:24.208326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeba9c2-b01b-40df-8ee6-5a6af502ab2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:24.213521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 847/1682] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:24.239963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bfeba9c2-b01b-40df-8ee6-5a6af502ab2d None None] MFA Rules not processed for user `2c917c21c7974c8d9746e8b05d9c8b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:24.241206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bfeba9c2-b01b-40df-8ee6-5a6af502ab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:24.270360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 836/1683] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:24.281457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:24.282952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:24.337396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276245d-fb59-477f-968e-9c1111fdb731 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:24.337688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276245d-fb59-477f-968e-9c1111fdb731 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:24.338063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276245d-fb59-477f-968e-9c1111fdb731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:24.372862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:24.428137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8276245d-fb59-477f-968e-9c1111fdb731 None None] MFA Rules not processed for user `182820d5c8f34678936cfcd354b5115a`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:24.432111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8276245d-fb59-477f-968e-9c1111fdb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:24.494887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5897289-ed5a-4050-95c2-d3e6cb70b29d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:24.517013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:24.518188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:24.518313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:24.518492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:24.540440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:24.542156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:24.562520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 848/1684] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:24.646779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 837/1685] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:24 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:24.649635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:24.651144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:24.680373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9562901-2aef-45dc-9936-16f13358f154 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:24.699246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:24.700184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:24.700184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:24.700433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:24.701202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:24.703356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:24.820056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 849/1686] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:24 2022] POST /identity/v3/projects => generated 403 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:43:24.824393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276245d-fb59-477f-968e-9c1111fdb731 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:24.841228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276245d-fb59-477f-968e-9c1111fdb731 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:24.957170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:24.957816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:24.958104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:24.958382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:24.959029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:24.961031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:25.011504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 838/1687] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:24 2022] POST /identity/v3/users => generated 308 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:25.013189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:25.026554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d03150-8a5e-4c7b-8888-cf4d5be912ce admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:25.062753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:25.062753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.062753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.062753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.063516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:25.065137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:25.098211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 850/1688] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:25.102261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:25.114012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a618dcd-74af-417c-81a9-ad3a3e75c216 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:25.124860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:25.125620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.126004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.126181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] PATH_INFO: `/v3/projects/a3e32ad320a543dea1878f1c16efa9c6/users/e342ad756a814f35b3f3e09b91ecdb71/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.159348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e32ad320a543dea1878f1c16efa9c6, user_id=e342ad756a814f35b3f3e09b91ecdb71, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:25.162969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:25.189152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 839/1689] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:25 2022] PUT /identity/v3/projects/a3e32ad320a543dea1878f1c16efa9c6/users/e342ad756a814f35b3f3e09b91ecdb71/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:25.193239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:25.222492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48b4de-b68d-411c-94aa-13fe0db4b097 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:25.365540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8b0d9-f3d3-49e2-8860-3ada65bf61be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:25.366395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8b0d9-f3d3-49e2-8860-3ada65bf61be admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.366764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8b0d9-f3d3-49e2-8860-3ada65bf61be admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.367116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8b0d9-f3d3-49e2-8860-3ada65bf61be admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.367901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8b0d9-f3d3-49e2-8860-3ada65bf61be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:25.369811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8b0d9-f3d3-49e2-8860-3ada65bf61be admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:25.384491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 851/1690] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:25.388480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:25.401992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1d5a2-a396-48e3-916c-7b4492f542a5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:25.407609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d818b-c359-4da0-89b2-0bf89a534c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:25.408642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d818b-c359-4da0-89b2-0bf89a534c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.409128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d818b-c359-4da0-89b2-0bf89a534c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.409602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d818b-c359-4da0-89b2-0bf89a534c7f admin admin] PATH_INFO: `/v3/projects/a3e32ad320a543dea1878f1c16efa9c6/users/e342ad756a814f35b3f3e09b91ecdb71/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.412000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d818b-c359-4da0-89b2-0bf89a534c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e32ad320a543dea1878f1c16efa9c6, user_id=e342ad756a814f35b3f3e09b91ecdb71, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:25.415213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d818b-c359-4da0-89b2-0bf89a534c7f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:25.423949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:25.432510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 840/1691] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:25 2022] PUT /identity/v3/projects/a3e32ad320a543dea1878f1c16efa9c6/users/e342ad756a814f35b3f3e09b91ecdb71/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:25.763112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77589c8-0faf-4e0a-a07c-ed66b6bb4444 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.763495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77589c8-0faf-4e0a-a07c-ed66b6bb4444 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.763846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77589c8-0faf-4e0a-a07c-ed66b6bb4444 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.765610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 852/1692] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 06:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:43:25.783140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.783566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.783923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.827578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77589c8-0faf-4e0a-a07c-ed66b6bb4444 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:25.829708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b77589c8-0faf-4e0a-a07c-ed66b6bb4444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:25.922448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] MFA Rules not processed for user `2fc4592ef54b41059fa7767a5ca3aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:25.924660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.036892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77589c8-0faf-4e0a-a07c-ed66b6bb4444 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.056666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.057377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.065527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.065822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] PATH_INFO: `/v3/projects/c484a5dd3cb142b99557d236e1aa4c96` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.067548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c484a5dd3cb142b99557d236e1aa4c96)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.069510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.102766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:26.127452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 841/1693] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:26.130005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.132964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.155667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c484a5dd3cb142b99557d236e1aa4c96', 'request_id': 'req-72839bd7-78e2-48af-b2fe-977c6eb80ed8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.167848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c543db1-fd2f-4d97-832d-52cbd64c63b2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.171578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c484a5dd3cb142b99557d236e1aa4c96', 'request_id': 'req-72839bd7-78e2-48af-b2fe-977c6eb80ed8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.188023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c484a5dd3cb142b99557d236e1aa4c96', 'request_id': 'req-72839bd7-78e2-48af-b2fe-977c6eb80ed8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.230904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] The token cache is being invalidate because project c484a5dd3cb142b99557d236e1aa4c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:26.232176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.234425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.240958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 853/1694] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:25 2022] DELETE /identity/v3/projects/c484a5dd3cb142b99557d236e1aa4c96 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:26.242479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.247110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.323553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.325635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.326081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.326601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] PATH_INFO: `/v3/projects/4f5059a58c6d4674a160593e81a69494` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.328212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5059a58c6d4674a160593e81a69494)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.331823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.405780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839bd7-78e2-48af-b2fe-977c6eb80ed8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.410600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a659a-8583-4bda-af75-a6a46722a36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.412105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a659a-8583-4bda-af75-a6a46722a36f service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.415229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a659a-8583-4bda-af75-a6a46722a36f service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.421351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a659a-8583-4bda-af75-a6a46722a36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.428822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-751a659a-8583-4bda-af75-a6a46722a36f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.460407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a659a-8583-4bda-af75-a6a46722a36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.466577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5059a58c6d4674a160593e81a69494', 'request_id': 'req-8fc89a30-3396-4175-b57b-91a50da715ae'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.467152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a659a-8583-4bda-af75-a6a46722a36f service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.484537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5059a58c6d4674a160593e81a69494', 'request_id': 'req-8fc89a30-3396-4175-b57b-91a50da715ae'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.492851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 854/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:26.498527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6c64-a5c1-4b0d-9d01-f886833c1c69 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.500149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6c64-a5c1-4b0d-9d01-f886833c1c69 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.500452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6c64-a5c1-4b0d-9d01-f886833c1c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.505109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5059a58c6d4674a160593e81a69494', 'request_id': 'req-8fc89a30-3396-4175-b57b-91a50da715ae'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.542514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] The token cache is being invalidate because project 4f5059a58c6d4674a160593e81a69494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:26.542887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.543717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.549827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 842/1696] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:26 2022] DELETE /identity/v3/projects/4f5059a58c6d4674a160593e81a69494 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:26.554590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.558593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.559912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-717a6c64-a5c1-4b0d-9d01-f886833c1c69 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:26.567297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-717a6c64-a5c1-4b0d-9d01-f886833c1c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.658600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:26.725356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc89a30-3396-4175-b57b-91a50da715ae admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.732727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.733409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.733697 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.733972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] PATH_INFO: `/v3/users/bdb38d96342b4c789b60fb0e320f70cb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.735239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb38d96342b4c789b60fb0e320f70cb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.737890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.756871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 855/1697] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:26.763093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef895765-2843-4a73-8a27-1bf7ccae81b2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.763482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef895765-2843-4a73-8a27-1bf7ccae81b2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.763966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef895765-2843-4a73-8a27-1bf7ccae81b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.814007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ef895765-2843-4a73-8a27-1bf7ccae81b2 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:26.815445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ef895765-2843-4a73-8a27-1bf7ccae81b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.836477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 856/1698] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:26.840108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef895765-2843-4a73-8a27-1bf7ccae81b2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.843089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb38d96342b4c789b60fb0e320f70cb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.856984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb38d96342b4c789b60fb0e320f70cb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.864772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef895765-2843-4a73-8a27-1bf7ccae81b2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.881180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.881790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.882078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.882358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.883038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:26.883443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.885152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.907229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb38d96342b4c789b60fb0e320f70cb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.908914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb38d96342b4c789b60fb0e320f70cb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.910768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb38d96342b4c789b60fb0e320f70cb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.918923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 857/1699] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:26.928247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.929887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.944399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94afe0-02b0-4d9d-aae4-9bd344a0fbf5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.961839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.964000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.964448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.965150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.965958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.970799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 843/1700] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:26 2022] DELETE /identity/v3/users/bdb38d96342b4c789b60fb0e320f70cb => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:26.971558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:27.031326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 858/1701] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:26 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:43:27.035765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:27.151867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa133b-d381-4117-8cb7-0c987e32d668 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:27.246757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:27.267569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99409a-3cdc-4710-807d-346d8b7abf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:27.268803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99409a-3cdc-4710-807d-346d8b7abf19 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.268803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99409a-3cdc-4710-807d-346d8b7abf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.268995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99409a-3cdc-4710-807d-346d8b7abf19 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.269731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99409a-3cdc-4710-807d-346d8b7abf19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:27.272409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99409a-3cdc-4710-807d-346d8b7abf19 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:27.340895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 844/1702] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:27.341895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:27.366135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679aa3d5-612c-4d67-aa46-1a27512ac7c2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:27.388020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:27.388663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.389288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.389714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.390546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:27.393926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:27.412019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:27.457975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5b9f-b5cd-4972-b5ef-ce94e7e4a7a2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.458612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5b9f-b5cd-4972-b5ef-ce94e7e4a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.459113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5b9f-b5cd-4972-b5ef-ce94e7e4a7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.466068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 859/1703] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:27.469834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:27.489034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ca684-ccf6-46a4-968c-c0622f749fe6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:27.502155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:27.503182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.507229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.508224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.508938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:27.513784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:27.529400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-00fe5b9f-b5cd-4972-b5ef-ce94e7e4a7a2 None None] MFA Rules not processed for user `7f3061468e644588a9f7f8de2af94827`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:27.530906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-00fe5b9f-b5cd-4972-b5ef-ce94e7e4a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:27.533957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 860/1704] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:27.536728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:27.559531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77c54-b200-4bb6-b673-61cfcc6ee7b4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:27.574900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf9590-5b41-4458-974c-233248f9210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:27.574900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9590-5b41-4458-974c-233248f9210e admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.574900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9590-5b41-4458-974c-233248f9210e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.574900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf9590-5b41-4458-974c-233248f9210e admin admin] PATH_INFO: `/v3/projects/228e1c31699e44aab3a6a95eaeb7eeec/users/c05327c4e7714283a226c1ea838de31a/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.626460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf9590-5b41-4458-974c-233248f9210e admin admin] RBAC: Authorizing `identity:create_grant(project_id=228e1c31699e44aab3a6a95eaeb7eeec, user_id=c05327c4e7714283a226c1ea838de31a, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:27.629299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf9590-5b41-4458-974c-233248f9210e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:27.676117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 861/1705] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:27 2022] PUT /identity/v3/projects/228e1c31699e44aab3a6a95eaeb7eeec/users/c05327c4e7714283a226c1ea838de31a/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:27.678087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a602a2-6bbb-49d2-9d7e-fee80e220526 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.678462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a602a2-6bbb-49d2-9d7e-fee80e220526 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.678747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a602a2-6bbb-49d2-9d7e-fee80e220526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.736957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 845/1706] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:27.749357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-44a602a2-6bbb-49d2-9d7e-fee80e220526 None None] MFA Rules not processed for user `c05327c4e7714283a226c1ea838de31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:27.749559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f53ad7-7944-4962-b731-163cd645b33d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.749953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f53ad7-7944-4962-b731-163cd645b33d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.750638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-44a602a2-6bbb-49d2-9d7e-fee80e220526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:27.751150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f53ad7-7944-4962-b731-163cd645b33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.777572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b4f53ad7-7944-4962-b731-163cd645b33d None None] MFA Rules not processed for user `7f3061468e644588a9f7f8de2af94827`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:27.780374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b4f53ad7-7944-4962-b731-163cd645b33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:27.885873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 862/1707] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:27.887810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5fdcc-8e47-4f7f-ad8b-70194188e18f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.888398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5fdcc-8e47-4f7f-ad8b-70194188e18f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.888984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5fdcc-8e47-4f7f-ad8b-70194188e18f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.890578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 863/1708] 10.222.0.52 () {54 vars in 961 bytes} [Wed Aug 24 06:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:27.891532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5fdcc-8e47-4f7f-ad8b-70194188e18f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:27.924968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 846/1709] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:27.934969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.935880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.936627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.974041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:27.975582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.047960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5fdcc-8e47-4f7f-ad8b-70194188e18f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:28.062724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:28.063398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.063681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.063956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.065302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:28.082558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:28.152034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 847/1710] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:28.152034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:28.152034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.153847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 864/1711] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/projects => generated 365 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:28.155107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:28.168853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.185549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:28.292554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:28.343973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53aec0-0648-4f7f-88e1-d1ccf907818e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:28.355581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6366-3257-4b5f-9d21-7a7c5788322b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:28.368686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:28.369387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.369672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.369947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] PATH_INFO: `/v3/projects/ec6201927fdb4ea29caad607e7d51da8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.373235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6201927fdb4ea29caad607e7d51da8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:28.375372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:28.476596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6201927fdb4ea29caad607e7d51da8', 'request_id': 'req-12c6afa7-a6fa-47f1-88a8-5190b4975120'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:28.486975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6201927fdb4ea29caad607e7d51da8', 'request_id': 'req-12c6afa7-a6fa-47f1-88a8-5190b4975120'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:28.506670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6201927fdb4ea29caad607e7d51da8', 'request_id': 'req-12c6afa7-a6fa-47f1-88a8-5190b4975120'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:28.524436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942c3be-956e-4d25-94da-75934e828e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:28.527041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942c3be-956e-4d25-94da-75934e828e46 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.527587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942c3be-956e-4d25-94da-75934e828e46 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.528014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942c3be-956e-4d25-94da-75934e828e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.547748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] The token cache is being invalidate because project ec6201927fdb4ea29caad607e7d51da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:28.550564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:28.554225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:28.566059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 865/1712] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:28 2022] DELETE /identity/v3/projects/ec6201927fdb4ea29caad607e7d51da8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:28.570462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:28.584216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.725591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942c3be-956e-4d25-94da-75934e828e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:28.727505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942c3be-956e-4d25-94da-75934e828e46 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:28.741921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 848/1713] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:28.742136 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6afa7-a6fa-47f1-88a8-5190b4975120 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:28.744298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b949b7-9da9-4f3b-a6d4-2bd2130d7070 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.744982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b949b7-9da9-4f3b-a6d4-2bd2130d7070 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.745959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b949b7-9da9-4f3b-a6d4-2bd2130d7070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.748440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:28.749427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.749587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.749952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.750661 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:28.753049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:28.776239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-36b949b7-9da9-4f3b-a6d4-2bd2130d7070 None None] MFA Rules not processed for user `2c917c21c7974c8d9746e8b05d9c8b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:28.777876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-36b949b7-9da9-4f3b-a6d4-2bd2130d7070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.792166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 866/1714] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:28 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:28.793680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:28.796364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.834790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:28.881338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 849/1715] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:28.881338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a10c8-2e3b-4618-b251-d66dd0e10637 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.881338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a10c8-2e3b-4618-b251-d66dd0e10637 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.881338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a10c8-2e3b-4618-b251-d66dd0e10637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.897959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:28.918403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f53a10c8-2e3b-4618-b251-d66dd0e10637 None None] MFA Rules not processed for user `f264e354671d4456bb33668fb11d9636`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:28.919924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f53a10c8-2e3b-4618-b251-d66dd0e10637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.938813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d39ab-25d3-46f2-90e1-64517c36749e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:28.952625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dd49f-9a31-4b16-9230-b0ed515476b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:28.953269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dd49f-9a31-4b16-9230-b0ed515476b6 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.953551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dd49f-9a31-4b16-9230-b0ed515476b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.953826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dd49f-9a31-4b16-9230-b0ed515476b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.959753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-251dd49f-9a31-4b16-9230-b0ed515476b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:28.986176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dd49f-9a31-4b16-9230-b0ed515476b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:28.989783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dd49f-9a31-4b16-9230-b0ed515476b6 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:29.018760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 867/1716] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:29.024552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ed21-5574-41c4-9984-74411b869554 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.024843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ed21-5574-41c4-9984-74411b869554 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.025124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ed21-5574-41c4-9984-74411b869554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.034569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 850/1717] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:29.035988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50862999-ed24-4acb-863a-330901a83228 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.036283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50862999-ed24-4acb-863a-330901a83228 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.036586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50862999-ed24-4acb-863a-330901a83228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.068109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8081ed21-5574-41c4-9984-74411b869554 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:29.069330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8081ed21-5574-41c4-9984-74411b869554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:29.080357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 868/1718] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:29.081864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad4987-17d7-4b65-ac0b-f51fd2545096 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.082151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad4987-17d7-4b65-ac0b-f51fd2545096 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.082428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad4987-17d7-4b65-ac0b-f51fd2545096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.120661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-50862999-ed24-4acb-863a-330901a83228 None None] MFA Rules not processed for user `e342ad756a814f35b3f3e09b91ecdb71`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:29.121913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-50862999-ed24-4acb-863a-330901a83228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:29.141132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-47ad4987-17d7-4b65-ac0b-f51fd2545096 None None] MFA Rules not processed for user `1d26cf886087408097cc2e435a785454`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:29.142366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-47ad4987-17d7-4b65-ac0b-f51fd2545096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:29.312910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 851/1719] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:29 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:29.317189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b46a50-1bc9-47b8-834f-4a1bc8157dde None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.330780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b46a50-1bc9-47b8-834f-4a1bc8157dde None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.334138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b46a50-1bc9-47b8-834f-4a1bc8157dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.401023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 869/1720] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:29.405517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad4987-17d7-4b65-ac0b-f51fd2545096 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:29.414472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-69b46a50-1bc9-47b8-834f-4a1bc8157dde None None] MFA Rules not processed for user `bfb5dc1fa16f465ba1ebca000b665d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:29.417875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-69b46a50-1bc9-47b8-834f-4a1bc8157dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:29.432175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad4987-17d7-4b65-ac0b-f51fd2545096 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:29.443951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3c7d9-b263-4be1-a009-eb714511dd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:29.446080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c7d9-b263-4be1-a009-eb714511dd82 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.446606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c7d9-b263-4be1-a009-eb714511dd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.447016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3c7d9-b263-4be1-a009-eb714511dd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.449035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3c7d9-b263-4be1-a009-eb714511dd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:29.454039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3c7d9-b263-4be1-a009-eb714511dd82 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:29.484381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 870/1721] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:29.487483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdeef8-a3e1-40ed-bdcd-321f748c870a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.488264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdeef8-a3e1-40ed-bdcd-321f748c870a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.489159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdeef8-a3e1-40ed-bdcd-321f748c870a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.579073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0cfdeef8-a3e1-40ed-bdcd-321f748c870a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:29.587390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0cfdeef8-a3e1-40ed-bdcd-321f748c870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:29.618536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 871/1722] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:29.631749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdeef8-a3e1-40ed-bdcd-321f748c870a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:29.684553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdeef8-a3e1-40ed-bdcd-321f748c870a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:29.693293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 852/1723] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:29.700055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be309c3-b6a3-448e-853a-43d4c5e06f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:29.701080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be309c3-b6a3-448e-853a-43d4c5e06f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.701853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be309c3-b6a3-448e-853a-43d4c5e06f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.702245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be309c3-b6a3-448e-853a-43d4c5e06f22 admin admin] PATH_INFO: `/v3/projects/8859f53a3ab642358bf2dea531672b21/users/f9f0f1550f0c4994af01b2719b117726/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.704045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b46a50-1bc9-47b8-834f-4a1bc8157dde None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:29.731035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b46a50-1bc9-47b8-834f-4a1bc8157dde None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:29.734073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be309c3-b6a3-448e-853a-43d4c5e06f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8859f53a3ab642358bf2dea531672b21, user_id=f9f0f1550f0c4994af01b2719b117726, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:29.738952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be309c3-b6a3-448e-853a-43d4c5e06f22 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:29.741852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cca1f-85bf-4da6-8794-5063e7d98596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:29.744396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cca1f-85bf-4da6-8794-5063e7d98596 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.744780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cca1f-85bf-4da6-8794-5063e7d98596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.746132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cca1f-85bf-4da6-8794-5063e7d98596 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.746836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cca1f-85bf-4da6-8794-5063e7d98596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:29.747223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cca1f-85bf-4da6-8794-5063e7d98596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:29.748959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cca1f-85bf-4da6-8794-5063e7d98596 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:29.768259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 872/1724] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:29 2022] PUT /identity/v3/projects/8859f53a3ab642358bf2dea531672b21/users/f9f0f1550f0c4994af01b2719b117726/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:29.770737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c948c74-8781-44af-95d8-be6915b2e4ef None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.771872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:29.772010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c948c74-8781-44af-95d8-be6915b2e4ef None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.773474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c948c74-8781-44af-95d8-be6915b2e4ef None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.778838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 873/1725] 10.222.0.52 () {54 vars in 912 bytes} [Wed Aug 24 06:43:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:29.781735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c948c74-8781-44af-95d8-be6915b2e4ef None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:29.788513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 853/1726] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:29.791786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6f87e-d4ee-43ca-916e-9c9ae74a6c58 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.792076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6f87e-d4ee-43ca-916e-9c9ae74a6c58 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.792349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6f87e-d4ee-43ca-916e-9c9ae74a6c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.850031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b0c6f87e-d4ee-43ca-916e-9c9ae74a6c58 None None] MFA Rules not processed for user `2fc4592ef54b41059fa7767a5ca3aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:29.851446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b0c6f87e-d4ee-43ca-916e-9c9ae74a6c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:29.909716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c948c74-8781-44af-95d8-be6915b2e4ef None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:29.914970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:29.916098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:29.916677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:29.917151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:29.917981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:29.919852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:29.955003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 874/1727] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:29.957923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:29.980601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c868943-a79d-451f-a1d8-2fb637891b87 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:29.992566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8b49a-f6df-45ee-8429-aaacb2371f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:29.994797 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 854/1728] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:29.995959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6f87e-d4ee-43ca-916e-9c9ae74a6c58 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:30.003756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8b49a-f6df-45ee-8429-aaacb2371f74 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.004398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8b49a-f6df-45ee-8429-aaacb2371f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.007597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8b49a-f6df-45ee-8429-aaacb2371f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.009802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8b49a-f6df-45ee-8429-aaacb2371f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:30.026619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8b49a-f6df-45ee-8429-aaacb2371f74 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:30.039538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6f87e-d4ee-43ca-916e-9c9ae74a6c58 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:30.054474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:30.056921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.057810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.059523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] PATH_INFO: `/v3/projects/8859f53a3ab642358bf2dea531672b21/users/f9f0f1550f0c4994af01b2719b117726/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.066886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8859f53a3ab642358bf2dea531672b21, user_id=f9f0f1550f0c4994af01b2719b117726, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:30.076808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:30.109073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 855/1729] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:29 2022] PUT /identity/v3/projects/8859f53a3ab642358bf2dea531672b21/users/f9f0f1550f0c4994af01b2719b117726/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:30.111056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:30.113380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:30.119418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:30.127625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 875/1730] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:29 2022] POST /identity/v3/projects => generated 393 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:30.128909 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c826061-4cca-4c6c-8907-ad17373489d7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.133220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c826061-4cca-4c6c-8907-ad17373489d7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.133509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c826061-4cca-4c6c-8907-ad17373489d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.236186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4c826061-4cca-4c6c-8907-ad17373489d7 None None] MFA Rules not processed for user `f9f0f1550f0c4994af01b2719b117726`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:30.239823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4c826061-4cca-4c6c-8907-ad17373489d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:30.258389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee33922-f86f-4096-947a-ff24151eef57 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:30.486128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:30.487511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.487729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.491168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.495996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 876/1731] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:30.501863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c826061-4cca-4c6c-8907-ad17373489d7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:30.534389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:30.536204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:30.562295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 856/1732] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:30.563437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:30.566203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:30.674522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c826061-4cca-4c6c-8907-ad17373489d7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:30.691323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:30.691966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.692469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.693755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.696928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:30.702038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:30.772802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb4c3f-6e62-459d-8405-ce430d0b183a service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:30.787815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 877/1733] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:30 2022] POST /identity/v3/users => generated 297 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:30.788837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:30.791418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:30.792630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.792931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.793214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.821707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:30.822223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96f270-a996-4c47-80be-072949030146 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:30.828939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:30.830442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:30.843618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:30.844264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.844552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.844844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.845512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:30.847155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:30.897217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 878/1734] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:30.902098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 857/1735] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:30.906447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:30.945270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b81fa-bf83-4e68-96f7-4bc5476fbcd3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:30.967127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:30.970866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:30.971271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:30.976649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:30.977367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:30.979089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:31.034573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 879/1736] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:31.036890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:31.071299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:31.077482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d773e-533b-4b4f-b555-e3f30e360f2b service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:31.092063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8aa714-f27a-4392-8f62-f95ffc007823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:31.092776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aa714-f27a-4392-8f62-f95ffc007823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:31.093065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aa714-f27a-4392-8f62-f95ffc007823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:31.093496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aa714-f27a-4392-8f62-f95ffc007823 admin admin] PATH_INFO: `/v3/projects/1cec3f1c29764a24932c7573b5ff30ec/users/3c8f205ebcdf4b339b67bc814bdf46fb/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:31.095021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647576c6-6ecf-450d-9c92-9c39591b089d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:31.120452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:31.121127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:31.121415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:31.121697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:31.125981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8aa714-f27a-4392-8f62-f95ffc007823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cec3f1c29764a24932c7573b5ff30ec, user_id=3c8f205ebcdf4b339b67bc814bdf46fb, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:31.128702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8aa714-f27a-4392-8f62-f95ffc007823 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:31.132999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:31.160810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 858/1737] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:31 2022] PUT /identity/v3/projects/1cec3f1c29764a24932c7573b5ff30ec/users/3c8f205ebcdf4b339b67bc814bdf46fb/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:31.171591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a1e04-672c-4f1b-9118-aa1901451ac3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:31.171885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a1e04-672c-4f1b-9118-aa1901451ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:31.172172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a1e04-672c-4f1b-9118-aa1901451ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:31.244082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1f6a1e04-672c-4f1b-9118-aa1901451ac3 None None] MFA Rules not processed for user `3c8f205ebcdf4b339b67bc814bdf46fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:31.246092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1f6a1e04-672c-4f1b-9118-aa1901451ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:31.272306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:31.299718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:31.301459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:31.323549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 880/1738] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:31.552024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 859/1739] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:31.558723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:31.582642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c781e-6df5-48b2-951c-903c459dbeec service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:31.803907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6d63-8038-4f8a-ae55-085f7f99fd64 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:31.804488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6d63-8038-4f8a-ae55-085f7f99fd64 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:31.804488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6d63-8038-4f8a-ae55-085f7f99fd64 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:31.806294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 860/1740] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:43:31.818646 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:31.819729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:31.820346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:31.840323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:31.840984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:31.841276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:31.841612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:31.842277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:31.849921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:31.892593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:31.893845 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:31.900435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 881/1741] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:31 2022] POST /identity/v3/projects => generated 391 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:31.905532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:32.157707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696128d-9968-43ad-8069-5ca95584291e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:32.167676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 861/1742] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:43:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:32.168728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:32.170216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:32.171509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:32.172230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:32.172890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:32.173232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:32.173997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:32.179807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:32.300028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 882/1743] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:31 2022] POST /identity/v3/users => generated 302 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:32.304063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:32.309086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:32.354567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c66e81-8856-4779-be96-48371ada8933 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:32.375246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:32.389024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:32.393181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:32.395347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:32.395925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:32.419845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:32.433419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:32.440770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634ee15-6159-44a8-841b-10c0799ddcd8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:32.496244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 883/1744] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:32.497961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:32.542492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:32.554578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:32.559460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:32.565087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:32.576346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28573ae5-665d-447b-9583-20c4044d78d7 service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:32.609436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:32.614216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:32.614641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:32.614929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:32.616178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:32.617326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:32.667825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 884/1745] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:32.668319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:32.674803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:32.769353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2e44-aea6-4757-a41b-9ede33c025f4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:32.800059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:32.806375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:32.806375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:32.807929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:32.842212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:32.874442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:32.902444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 862/1746] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:32.903626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:32.971221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16267acc-23f5-40cd-a7f7-f2fc5c44ccac service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:32.999815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed11077-37cc-4849-9ba6-e24d251efd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:33.000678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed11077-37cc-4849-9ba6-e24d251efd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:33.002053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed11077-37cc-4849-9ba6-e24d251efd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:33.005205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed11077-37cc-4849-9ba6-e24d251efd73 admin admin] PATH_INFO: `/v3/projects/8b19f0745b334c7e8cdd49f1e1d55f42/users/09c87642ce6241049c6f518456549f9b/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:33.064236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed11077-37cc-4849-9ba6-e24d251efd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b19f0745b334c7e8cdd49f1e1d55f42, user_id=09c87642ce6241049c6f518456549f9b, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:33.072838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed11077-37cc-4849-9ba6-e24d251efd73 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:33.090346 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:33.123766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 863/1747] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:32 2022] PUT /identity/v3/projects/8b19f0745b334c7e8cdd49f1e1d55f42/users/09c87642ce6241049c6f518456549f9b/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:33.136067 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523a8e7-62df-4e49-8660-57861cd29972 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:33.142390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523a8e7-62df-4e49-8660-57861cd29972 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:33.148400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523a8e7-62df-4e49-8660-57861cd29972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:33.241455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:33.243102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:33.302563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 885/1748] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:32 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:33.303933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:33.344132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80026569-e9bf-4d78-9c30-d3fb1c6e3626 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:33.381096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5523a8e7-62df-4e49-8660-57861cd29972 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:33.385142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5523a8e7-62df-4e49-8660-57861cd29972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:33.620404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:33.621176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:33.621343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:33.621624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:33.622298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:33.635714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:33.684067 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 886/1749] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:33.688596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:33.731396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f054b01-c221-4572-8501-706872ce7a92 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:33.749740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:33.750449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:33.750739 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:33.751021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] PATH_INFO: `/v3/projects/8b19f0745b334c7e8cdd49f1e1d55f42/users/09c87642ce6241049c6f518456549f9b/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:33.752431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 864/1750] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 628 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:33.759763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b19f0745b334c7e8cdd49f1e1d55f42, user_id=09c87642ce6241049c6f518456549f9b, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:33.762834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:33.809321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 887/1751] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:33 2022] PUT /identity/v3/projects/8b19f0745b334c7e8cdd49f1e1d55f42/users/09c87642ce6241049c6f518456549f9b/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:33.819711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:33.820139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:33.820514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:33.913645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] MFA Rules not processed for user `09c87642ce6241049c6f518456549f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:33.915779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:33.967461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:34.069057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 865/1752] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:34.077473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:34.131725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:34.133735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:34.366717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bf05b-731e-46ae-afcc-1cac50e99c4c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:34.378539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:34.382848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:34.383676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:34.384665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:34.413276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:34.417587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:34.453484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 888/1753] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:34 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:34.512897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4aea1-0492-4f2d-9638-ec8dfae9d610 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:34.514736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:34.516693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:34.535217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ef580-43c3-487e-8ef2-08b6f288cd09 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:34.537311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:34.538216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:34.539028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:34.540686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:34.546654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:34.547762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:34.548061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:34.551029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:34.703131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:34.708911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:34.710833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:34.714953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:34.731103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 866/1754] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:43:34.734991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 889/1755] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:35.060235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:35.063012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:35.272297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:35.411555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a49b0e-952f-4452-bb16-3c5d6fb633f0 service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:35.456361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:35.457044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:35.457336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:35.463912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] PATH_INFO: `/v3/users/182820d5c8f34678936cfcd354b5115a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:35.474580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182820d5c8f34678936cfcd354b5115a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:35.476740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:35.754647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182820d5c8f34678936cfcd354b5115a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:35.772899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182820d5c8f34678936cfcd354b5115a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:35.793048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182820d5c8f34678936cfcd354b5115a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:35.793855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182820d5c8f34678936cfcd354b5115a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:35.794643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182820d5c8f34678936cfcd354b5115a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:35.858389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 867/1756] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:35 2022] DELETE /identity/v3/users/182820d5c8f34678936cfcd354b5115a => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:37.821041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:37.831565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:37.868050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ade5ef-ab7f-4c8c-a200-be5a2c6ccfb6 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:38.095817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:38.096540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:38.096835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:38.097110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] PATH_INFO: `/v3/projects/c7c4e6c97e5842f29be38e8de496b5e7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:38.100366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c4e6c97e5842f29be38e8de496b5e7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:38.101732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:38.254607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c4e6c97e5842f29be38e8de496b5e7', 'request_id': 'req-36c19e88-f8d5-4ef2-a10c-22d72401084c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:38.273446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c4e6c97e5842f29be38e8de496b5e7', 'request_id': 'req-36c19e88-f8d5-4ef2-a10c-22d72401084c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:38.294703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c4e6c97e5842f29be38e8de496b5e7', 'request_id': 'req-36c19e88-f8d5-4ef2-a10c-22d72401084c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:38.328848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] The token cache is being invalidate because project c7c4e6c97e5842f29be38e8de496b5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:38.329202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:38.330364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-36c19e88-f8d5-4ef2-a10c-22d72401084c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:38.336847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 890/1757] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:37 2022] DELETE /identity/v3/projects/c7c4e6c97e5842f29be38e8de496b5e7 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:38.341547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:38.347037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:38.420821 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569820a-dda1-41aa-a5e0-541f060608a7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:38.734530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:38.735791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:38.737609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:38.742941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] PATH_INFO: `/v3/users/e342ad756a814f35b3f3e09b91ecdb71` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:38.745767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e342ad756a814f35b3f3e09b91ecdb71)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:38.748219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:39.021539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e342ad756a814f35b3f3e09b91ecdb71'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:39.047839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e342ad756a814f35b3f3e09b91ecdb71'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:39.076573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e342ad756a814f35b3f3e09b91ecdb71'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:39.077953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e342ad756a814f35b3f3e09b91ecdb71'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:39.079021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e342ad756a814f35b3f3e09b91ecdb71'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:39.121506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:39.182827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 868/1758] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:38 2022] DELETE /identity/v3/users/e342ad756a814f35b3f3e09b91ecdb71 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:40.968697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:40.969279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:40.973242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:41.056216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] MFA Rules not processed for user `2fc4592ef54b41059fa7767a5ca3aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:41.057747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:41.243778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 891/1759] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 24 06:43:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:41.271357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:41.273858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:41.504730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:41.507009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:41.508414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b069488-2652-4394-9f8e-9af9e4fd6a50 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:41.517537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:41.518288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:41.518906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:41.519320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] PATH_INFO: `/v3/users/3c8f205ebcdf4b339b67bc814bdf46fb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:41.522263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8f205ebcdf4b339b67bc814bdf46fb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:41.538509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:41.540682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52513f31-15ed-4783-970b-94927516b1f1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:41.553881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:41.556871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:41.558835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:41.563117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:41.608944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:41.613897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:41.665341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 892/1760] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:41.678338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:41.680209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:41.724825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b31358-6e5b-411f-8dd9-dc3de6ea9c02 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:41.738300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:41.738939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:41.743859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:41.744123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:41.746441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8f205ebcdf4b339b67bc814bdf46fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:41.766705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:41.768260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:41.778927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8f205ebcdf4b339b67bc814bdf46fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:41.805160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8f205ebcdf4b339b67bc814bdf46fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:41.806395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8f205ebcdf4b339b67bc814bdf46fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:41.807825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8f205ebcdf4b339b67bc814bdf46fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:41.827971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 893/1761] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:41.828813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:41.851241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1e51f-4489-4cbc-bf96-50c149fb3758 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:41.865719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11c678-ad11-4598-ae56-234cad2acbf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:41.865719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11c678-ad11-4598-ae56-234cad2acbf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:41.865914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11c678-ad11-4598-ae56-234cad2acbf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:41.866133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11c678-ad11-4598-ae56-234cad2acbf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:41.873020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-de11c678-ad11-4598-ae56-234cad2acbf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:41.888334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11c678-ad11-4598-ae56-234cad2acbf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:41.889965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11c678-ad11-4598-ae56-234cad2acbf5 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:41.897392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 869/1762] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:41 2022] DELETE /identity/v3/users/3c8f205ebcdf4b339b67bc814bdf46fb => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:41.897392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:41.902142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:41.913158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 894/1763] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:41.923270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc960d-2e57-46d2-af26-4f4fbd72f46a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:41.923914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc960d-2e57-46d2-af26-4f4fbd72f46a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:41.924521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc960d-2e57-46d2-af26-4f4fbd72f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:41.933632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:41.962107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c1bc960d-2e57-46d2-af26-4f4fbd72f46a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:41.964285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c1bc960d-2e57-46d2-af26-4f4fbd72f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:42.085742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f5197-b78b-47fe-a377-a5ce0fc9574a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:42.101345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 895/1764] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:42.118315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:42.119289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:42.119688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:42.119978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] PATH_INFO: `/v3/projects/a3e32ad320a543dea1878f1c16efa9c6` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:42.121191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e32ad320a543dea1878f1c16efa9c6)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:42.123114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:42.146156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:42.251542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e32ad320a543dea1878f1c16efa9c6', 'request_id': 'req-011275eb-b316-449f-b4ca-3e2cd19a6bed'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:42.261478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e32ad320a543dea1878f1c16efa9c6', 'request_id': 'req-011275eb-b316-449f-b4ca-3e2cd19a6bed'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:42.276677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e32ad320a543dea1878f1c16efa9c6', 'request_id': 'req-011275eb-b316-449f-b4ca-3e2cd19a6bed'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:42.293551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] The token cache is being invalidate because project a3e32ad320a543dea1878f1c16efa9c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:42.293852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:42.297374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:42.300997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 870/1765] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:41 2022] DELETE /identity/v3/projects/a3e32ad320a543dea1878f1c16efa9c6 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:42.305831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91afcd-e853-4879-86c8-05c1ff5c4fb5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:42.306228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91afcd-e853-4879-86c8-05c1ff5c4fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:42.312351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91afcd-e853-4879-86c8-05c1ff5c4fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:42.366198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:42.367698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:42.417190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6c91afcd-e853-4879-86c8-05c1ff5c4fb5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:42.418594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6c91afcd-e853-4879-86c8-05c1ff5c4fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:42.564056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011275eb-b316-449f-b4ca-3e2cd19a6bed admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:42.575709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:42.581900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:42.582352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:42.582630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:42.592583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:42.780062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 896/1766] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:42.786153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311e86-d3a2-4cd4-a1f3-7ef34c806f3d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:42.786363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311e86-d3a2-4cd4-a1f3-7ef34c806f3d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:42.786651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c311e86-d3a2-4cd4-a1f3-7ef34c806f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:42.852527 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:42.859092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:42.897071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 871/1767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:42.897522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8c311e86-d3a2-4cd4-a1f3-7ef34c806f3d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:42.904563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8c311e86-d3a2-4cd4-a1f3-7ef34c806f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:42.964673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 897/1768] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:42.968062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:42.982366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d08190-52a4-44cb-92d3-500a119891f8 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:42.993380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:42.995547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:42.995846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:42.996131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:42.998800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:42.999467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:43.001351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:43.028504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 872/1769] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:43.033114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311e86-d3a2-4cd4-a1f3-7ef34c806f3d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:43.068465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c311e86-d3a2-4cd4-a1f3-7ef34c806f3d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:43.091632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a28c71-f9be-4cde-bf20-d4e05a1238bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:43.102423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a28c71-f9be-4cde-bf20-d4e05a1238bb admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:43.103657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a28c71-f9be-4cde-bf20-d4e05a1238bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:43.103657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a28c71-f9be-4cde-bf20-d4e05a1238bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:43.105807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a28c71-f9be-4cde-bf20-d4e05a1238bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:43.111425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a28c71-f9be-4cde-bf20-d4e05a1238bb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:43.213795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 898/1770] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:43 2022] POST /identity/v3/projects => generated 355 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:43.215391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:43.236882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a455f-277c-43fc-b089-4af768fd6e79 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:43.258542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c0427-a09a-457a-8754-9e5c215aa18c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:43.258837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c0427-a09a-457a-8754-9e5c215aa18c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:43.259169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c0427-a09a-457a-8754-9e5c215aa18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:43.330063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a36c0427-a09a-457a-8754-9e5c215aa18c None None] MFA Rules not processed for user `1d26cf886087408097cc2e435a785454`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:43.331394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a36c0427-a09a-457a-8754-9e5c215aa18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:43.359439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:43.491534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:43.492294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:43.492611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:43.493888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:43.493888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:43.496060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:43.574520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 873/1771] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:43 2022] POST /identity/v3/users => generated 278 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:43.575660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:43.579047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 899/1772] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:43.612628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79300066-d733-4c2f-90b8-a05bf7495101 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:43.626594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:43.628315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:43.628606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:43.628932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:43.632928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:43.648862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:43.688669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 874/1773] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:43.689703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c0427-a09a-457a-8754-9e5c215aa18c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:43.707732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c0427-a09a-457a-8754-9e5c215aa18c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:43.722805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90dc99-583f-4d92-b1bd-09f314e4789f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:43.725103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90dc99-583f-4d92-b1bd-09f314e4789f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:43.726218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90dc99-583f-4d92-b1bd-09f314e4789f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:43.726547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90dc99-583f-4d92-b1bd-09f314e4789f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:43.728837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90dc99-583f-4d92-b1bd-09f314e4789f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:43.733172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90dc99-583f-4d92-b1bd-09f314e4789f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:43.767400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:43.777098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 900/1774] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:43.779641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9ea31-e3d5-4786-9c51-21a49b1b0c55 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:43.800490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:43.801737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:43.802312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:43.802831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] PATH_INFO: `/v3/projects/8cd7ef6de68d49b19fdf31e38b49d73b/users/b3a4dc2eb6634a9db57083a7ac819c47/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:43.826330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd7ef6de68d49b19fdf31e38b49d73b, user_id=b3a4dc2eb6634a9db57083a7ac819c47, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:43.830581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:43.860222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 875/1775] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:43 2022] PUT /identity/v3/projects/8cd7ef6de68d49b19fdf31e38b49d73b/users/b3a4dc2eb6634a9db57083a7ac819c47/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:43.873520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b0c38-a75c-4dc4-8f3e-be638ebf27b1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:43.873922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b0c38-a75c-4dc4-8f3e-be638ebf27b1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:43.874154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b0c38-a75c-4dc4-8f3e-be638ebf27b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:43.960995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:44.017350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ba7b0c38-a75c-4dc4-8f3e-be638ebf27b1 None None] MFA Rules not processed for user `b3a4dc2eb6634a9db57083a7ac819c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:44.025131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ba7b0c38-a75c-4dc4-8f3e-be638ebf27b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:44.283704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 901/1776] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:44.288094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:44.497788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf74c-1441-448c-9f89-ee952b5ad1a7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:44.520882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:44.521764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:44.522233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:44.522665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:44.524074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:44.526515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:44.607365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 876/1777] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:44 2022] POST /identity/v3/projects => generated 355 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:44.611051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b0c38-a75c-4dc4-8f3e-be638ebf27b1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:44.665858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b0c38-a75c-4dc4-8f3e-be638ebf27b1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:44.998938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:45.003031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:45.004419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:45.006050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:45.009855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:45.016110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:45.104285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 902/1778] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:44 2022] POST /identity/v3/users => generated 284 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:45.106665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:45.126961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e07bf-7aa6-4804-87f7-4067c74655ce admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:45.151994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:45.152682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:45.152969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:45.155312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:45.156042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:45.161088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:45.195818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 877/1779] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:45.199733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:45.227002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c77d6d-9984-447c-9b64-ee891918db8d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:45.257566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:45.258273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:45.262923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:45.263372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] PATH_INFO: `/v3/projects/86f01233e6624b45ac1306380839b2e5/users/4c3e79238cb24c0490db1d897baa6c7b/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:45.373807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f01233e6624b45ac1306380839b2e5, user_id=4c3e79238cb24c0490db1d897baa6c7b, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:45.406700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:45.467884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 903/1780] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:45 2022] PUT /identity/v3/projects/86f01233e6624b45ac1306380839b2e5/users/4c3e79238cb24c0490db1d897baa6c7b/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:45.472237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:45.496692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c5bab-88b6-41eb-a1f2-021ecf173ded admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:45.710266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:45.713011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:45.778165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:45.778953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:45.779224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:45.779499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:45.780169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:45.781849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:45.798890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:45.834527 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 878/1781] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:45.835748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:45.886620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601de1-5209-460b-888d-61d0a675f95d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:45.906338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03b693-81a6-4af2-a619-bb9934660b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:45.907186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03b693-81a6-4af2-a619-bb9934660b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:45.907571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03b693-81a6-4af2-a619-bb9934660b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:45.921823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03b693-81a6-4af2-a619-bb9934660b63 admin admin] PATH_INFO: `/v3/projects/86f01233e6624b45ac1306380839b2e5/users/4c3e79238cb24c0490db1d897baa6c7b/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:45.932270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03b693-81a6-4af2-a619-bb9934660b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f01233e6624b45ac1306380839b2e5, user_id=4c3e79238cb24c0490db1d897baa6c7b, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:45.955536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03b693-81a6-4af2-a619-bb9934660b63 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:46.017226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11523d9d-1102-4cfc-b1aa-08b05337614c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:46.024951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 879/1782] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:45 2022] PUT /identity/v3/projects/86f01233e6624b45ac1306380839b2e5/users/4c3e79238cb24c0490db1d897baa6c7b/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:46.026890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:46.027200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:46.027498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:46.062804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:46.183436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] MFA Rules not processed for user `4c3e79238cb24c0490db1d897baa6c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:46.184703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:46.250316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:46.254767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:46.254767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:46.256967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] PATH_INFO: `/v3/users/c05327c4e7714283a226c1ea838de31a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:46.264794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05327c4e7714283a226c1ea838de31a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:46.270599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:46.513997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05327c4e7714283a226c1ea838de31a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.515661 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05327c4e7714283a226c1ea838de31a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.536376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 880/1783] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:46.537426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:46.542199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:46.567921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05327c4e7714283a226c1ea838de31a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.574500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05327c4e7714283a226c1ea838de31a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.586495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7ee9b-39e7-4d14-bdf5-0d60af8ab833 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:46.605715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:46.606439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:46.606727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:46.607717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] PATH_INFO: `/v3/projects/1cec3f1c29764a24932c7573b5ff30ec` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:46.609051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cec3f1c29764a24932c7573b5ff30ec)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:46.611446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:46.614272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05327c4e7714283a226c1ea838de31a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.698182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 904/1784] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:45 2022] DELETE /identity/v3/users/c05327c4e7714283a226c1ea838de31a => generated 0 bytes in 988 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:46.700091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:46.703675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:46.752119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cec3f1c29764a24932c7573b5ff30ec', 'request_id': 'req-c5fe9e7b-71aa-4719-b794-0ce4475429ff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.760412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cec3f1c29764a24932c7573b5ff30ec', 'request_id': 'req-c5fe9e7b-71aa-4719-b794-0ce4475429ff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.769981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cec3f1c29764a24932c7573b5ff30ec', 'request_id': 'req-c5fe9e7b-71aa-4719-b794-0ce4475429ff'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.800518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] The token cache is being invalidate because project 1cec3f1c29764a24932c7573b5ff30ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:46.802923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.804785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c5fe9e7b-71aa-4719-b794-0ce4475429ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:46.808174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 881/1785] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:46 2022] DELETE /identity/v3/projects/1cec3f1c29764a24932c7573b5ff30ec => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:46.810903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:46.811415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:46.812648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:46.874327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:46.875237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c475d-73a1-4e04-9a9b-ae39a370583e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:46.878301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:46.980337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:46.994303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:47.002685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:47.004353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:47.004644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:47.004929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:47.007874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:47.012283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 882/1786] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:47.013849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:47.020373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:47.054321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17baa09-62a0-4eda-af21-156f9579b550 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:47.082271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:47.083115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:47.083625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:47.084041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] PATH_INFO: `/v3/users/09c87642ce6241049c6f518456549f9b` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:47.085932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c87642ce6241049c6f518456549f9b)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:47.091175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:47.206314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:47.208209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:47.232385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 905/1787] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:47.234734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c87642ce6241049c6f518456549f9b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:47.248965 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c87642ce6241049c6f518456549f9b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:47.266042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c87642ce6241049c6f518456549f9b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:47.267179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c87642ce6241049c6f518456549f9b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:47.268926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c87642ce6241049c6f518456549f9b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:47.271229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:47.273184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:47.361969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 883/1788] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:47 2022] DELETE /identity/v3/users/09c87642ce6241049c6f518456549f9b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:47.411365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:47.413091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:47.475703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d6fd4-cfd5-41fe-b574-c71fd85ec203 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:47.553172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7926a07-41cd-4e33-8832-7259771adff9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:47.560775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dedfe8-e36b-4a0a-b50e-30dfe67d3bc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:47.562402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dedfe8-e36b-4a0a-b50e-30dfe67d3bc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:47.562816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dedfe8-e36b-4a0a-b50e-30dfe67d3bc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:47.563398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dedfe8-e36b-4a0a-b50e-30dfe67d3bc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:47.565771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-68dedfe8-e36b-4a0a-b50e-30dfe67d3bc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:47.600247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1577-193f-49af-b0e1-2c52120c7a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:47.600894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1577-193f-49af-b0e1-2c52120c7a9b service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:47.601233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1577-193f-49af-b0e1-2c52120c7a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:47.601515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1577-193f-49af-b0e1-2c52120c7a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:47.608119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d82d1577-193f-49af-b0e1-2c52120c7a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:47.751054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dedfe8-e36b-4a0a-b50e-30dfe67d3bc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:47.757811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dedfe8-e36b-4a0a-b50e-30dfe67d3bc3 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:47.804794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 884/1789] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:47.809817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308450d-cea7-4594-9a92-2d4268e8580e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:47.810131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308450d-cea7-4594-9a92-2d4268e8580e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:47.810421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308450d-cea7-4594-9a92-2d4268e8580e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:47.871536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1577-193f-49af-b0e1-2c52120c7a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:47.878606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1577-193f-49af-b0e1-2c52120c7a9b service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:47.903752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d308450d-cea7-4594-9a92-2d4268e8580e None None] MFA Rules not processed for user `bfb5dc1fa16f465ba1ebca000b665d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:47.910023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d308450d-cea7-4594-9a92-2d4268e8580e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:47.926489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 906/1790] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:47.954064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:48.086416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 885/1791] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:48.720207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bfee6-bef9-46b1-84d9-7bc579901a38 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:48.721352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bfee6-bef9-46b1-84d9-7bc579901a38 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:48.721878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bfee6-bef9-46b1-84d9-7bc579901a38 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:48.726064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 907/1792] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:43:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:48.781996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe6866-82b4-4346-aa7d-9b709880e982 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:48.782408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe6866-82b4-4346-aa7d-9b709880e982 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:48.782700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe6866-82b4-4346-aa7d-9b709880e982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:48.834568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-eefe6866-82b4-4346-aa7d-9b709880e982 None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:48.844121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eefe6866-82b4-4346-aa7d-9b709880e982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:49.054481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 886/1793] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:43:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:49.292835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18218-15d9-4c55-9566-d667fd916bd6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:49.293123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18218-15d9-4c55-9566-d667fd916bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:49.293661 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf18218-15d9-4c55-9566-d667fd916bd6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:49.295692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 908/1794] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:49.449957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:49.450345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:49.450742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:49.520392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:49.525742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:49.566865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 887/1795] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:43:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:50.021487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18218-15d9-4c55-9566-d667fd916bd6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.023398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2bf18218-15d9-4c55-9566-d667fd916bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.035585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:50.041652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf18218-15d9-4c55-9566-d667fd916bd6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:50.049753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:50.050717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.051488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.051923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] PATH_INFO: `/v3/projects/8b19f0745b334c7e8cdd49f1e1d55f42` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.053474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b19f0745b334c7e8cdd49f1e1d55f42)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:50.055940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:50.131417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b19f0745b334c7e8cdd49f1e1d55f42', 'request_id': 'req-dab6dc56-131a-4f5d-826f-dc49551003ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.139015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b19f0745b334c7e8cdd49f1e1d55f42', 'request_id': 'req-dab6dc56-131a-4f5d-826f-dc49551003ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.145193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.146632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b19f0745b334c7e8cdd49f1e1d55f42', 'request_id': 'req-dab6dc56-131a-4f5d-826f-dc49551003ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.147604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.167274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] The token cache is being invalidate because project 8b19f0745b334c7e8cdd49f1e1d55f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:50.167522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.168220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dab6dc56-131a-4f5d-826f-dc49551003ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.172319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 909/1796] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:50 2022] DELETE /identity/v3/projects/8b19f0745b334c7e8cdd49f1e1d55f42 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:50.177715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.178175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.178660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.200388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367e12d-c32c-4468-bcd3-a3fdf990bbf9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:50.210044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:50.212391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.284319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 910/1797] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:50.285495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.295594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.336621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:50.338141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.347301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.347809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] PATH_INFO: `/v3/projects/228e1c31699e44aab3a6a95eaeb7eeec` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.350374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228e1c31699e44aab3a6a95eaeb7eeec)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:50.355334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:50.425290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd22c1-50e2-41b3-9b78-96f4a777e1fb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:50.439546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f860467-82d0-483c-a9de-b296d5f761c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:50.440415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f860467-82d0-483c-a9de-b296d5f761c0 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.440753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f860467-82d0-483c-a9de-b296d5f761c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.441818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f860467-82d0-483c-a9de-b296d5f761c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.444224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0f860467-82d0-483c-a9de-b296d5f761c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.469884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f860467-82d0-483c-a9de-b296d5f761c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:50.473234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f860467-82d0-483c-a9de-b296d5f761c0 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:50.481206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228e1c31699e44aab3a6a95eaeb7eeec', 'request_id': 'req-8c78a6eb-fac0-446c-ad15-200483611cb2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.485187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:50.496888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228e1c31699e44aab3a6a95eaeb7eeec', 'request_id': 'req-8c78a6eb-fac0-446c-ad15-200483611cb2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.507318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 911/1798] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:50.513090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.513614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.514161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.537120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228e1c31699e44aab3a6a95eaeb7eeec', 'request_id': 'req-8c78a6eb-fac0-446c-ad15-200483611cb2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.543803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:50.549780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.570627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] The token cache is being invalidate because project 228e1c31699e44aab3a6a95eaeb7eeec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:50.571176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.573970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.578382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 888/1799] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:50 2022] DELETE /identity/v3/projects/228e1c31699e44aab3a6a95eaeb7eeec => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:50.583102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.587157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.596667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:50.732621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a6eb-fac0-446c-ad15-200483611cb2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:50.742983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:50.744201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.744484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.744760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] PATH_INFO: `/v3/users/f9f0f1550f0c4994af01b2719b117726` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.747396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9f0f1550f0c4994af01b2719b117726)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:50.756510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:50.758484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 912/1800] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:50.760240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.761766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.917838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9f0f1550f0c4994af01b2719b117726'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.939181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9f0f1550f0c4994af01b2719b117726'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.957973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9f0f1550f0c4994af01b2719b117726'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.958392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9f0f1550f0c4994af01b2719b117726'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.959335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9f0f1550f0c4994af01b2719b117726'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.989109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 889/1801] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:50 2022] DELETE /identity/v3/users/f9f0f1550f0c4994af01b2719b117726 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:50.990388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.991919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:51.006123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199dfcd-44a7-4318-8f1e-afbe5b402a84 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:51.189479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62505565-d33f-4d52-a58d-73725be5bb70 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:51.193412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:51.195691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.201731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.202280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.214129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:51.218988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:51.224672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.225461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.225461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.230330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:51.230655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:51.244106 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:51.247551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:51.257664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:51.264560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:51.276391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 890/1802] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:51.284471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:51.304176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 913/1803] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:51.322218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2de7-1262-471c-98da-9182c39ef28c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:51.334710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:51.335227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.335519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.335914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.336621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:51.338644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:51.392857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 891/1804] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:51 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:51.396690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:51.418700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d4e18-e30b-4025-a25f-41e6d8f8230f service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:51.503579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:51.586972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:51.591464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.593167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.595488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.596474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:51.601812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:51.644055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 914/1805] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:51 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:43:51.650104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:51.663966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bfa37-158f-4e40-9986-5bd5acb7d91d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:51.669651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:51.670189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.670477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.670909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.671694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:51.674126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:51.691811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:51.704000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 892/1806] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:51.737052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f7a53-0c6a-4ecc-89dd-c791d063a411 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:51.743527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee728c-c8d3-4deb-82fe-9eb36dffef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:51.744012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee728c-c8d3-4deb-82fe-9eb36dffef8c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.744421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee728c-c8d3-4deb-82fe-9eb36dffef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.744806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee728c-c8d3-4deb-82fe-9eb36dffef8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.745797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee728c-c8d3-4deb-82fe-9eb36dffef8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:51.747550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee728c-c8d3-4deb-82fe-9eb36dffef8c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:51.762515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 915/1807] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:51.766664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:51.781424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180f90-4e14-449e-a3f8-c1bbb663a154 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:51.788399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:51.789694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.790178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.791087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] PATH_INFO: `/v3/projects/aaaed9f48803494a8e12c2cec1bc827a/users/2b6958bc50e94c1792191aa0adcc71cf/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.809189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaaed9f48803494a8e12c2cec1bc827a, user_id=2b6958bc50e94c1792191aa0adcc71cf, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:51.812385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:51.826654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 893/1808] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:51 2022] PUT /identity/v3/projects/aaaed9f48803494a8e12c2cec1bc827a/users/2b6958bc50e94c1792191aa0adcc71cf/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:51.837700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:51.838027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:51.838919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:51.874173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] MFA Rules not processed for user `2b6958bc50e94c1792191aa0adcc71cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:51.876108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:51.956879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 916/1809] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:51.962158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.065648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.068498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:52.099169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335665a0-dd90-42b5-8d3e-ad46c6dd2bb8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.106872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.107535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.107814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.108091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.108895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.111587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.145524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 894/1810] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:51 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:43:52.150496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.160017 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95066-83ca-40d2-9521-5932b46ed272 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.216505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:52.226157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcf245-5853-4c9c-924f-a16e09903fb0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.231951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.232816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.233318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.233482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.233650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] PATH_INFO: `/v3/projects/8859f53a3ab642358bf2dea531672b21` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.234178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.234400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.234772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.234845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8859f53a3ab642358bf2dea531672b21)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.235435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.236973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.237654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.268304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 895/1811] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:52 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:43:52.272533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.283274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de44ab-90b2-49e5-b7dd-f35db02ba4ee admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.296437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8859f53a3ab642358bf2dea531672b21', 'request_id': 'req-d6f97d7f-1221-48f3-8d36-524c219bce95'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:52.297807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:52.306308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8859f53a3ab642358bf2dea531672b21', 'request_id': 'req-d6f97d7f-1221-48f3-8d36-524c219bce95'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:52.315399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8859f53a3ab642358bf2dea531672b21', 'request_id': 'req-d6f97d7f-1221-48f3-8d36-524c219bce95'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:52.339296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] The token cache is being invalidate because project 8859f53a3ab642358bf2dea531672b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:52.339296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:52.340512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d6f97d7f-1221-48f3-8d36-524c219bce95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:52.343316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 917/1812] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:52 2022] DELETE /identity/v3/projects/8859f53a3ab642358bf2dea531672b21 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:52.347714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bb1fc-a54c-4b6d-ab8f-c1bfef94724f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.347714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bb1fc-a54c-4b6d-ab8f-c1bfef94724f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.347975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bb1fc-a54c-4b6d-ab8f-c1bfef94724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.370759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-094bb1fc-a54c-4b6d-ab8f-c1bfef94724f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:52.372147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-094bb1fc-a54c-4b6d-ab8f-c1bfef94724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:52.375427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.376170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.376462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.376738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.377518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.379456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.388673 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 918/1813] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:52.391017 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e8011-f755-4c50-95b1-6bd5e3408ea9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.391564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e8011-f755-4c50-95b1-6bd5e3408ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.392028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e8011-f755-4c50-95b1-6bd5e3408ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.398599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 896/1814] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:43:52.399848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.402890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:52.421614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2b6e8011-f755-4c50-95b1-6bd5e3408ea9 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:52.423232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2b6e8011-f755-4c50-95b1-6bd5e3408ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:52.434578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e861d-454a-4027-942e-dea31a678937 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.436994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 919/1815] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:52.437947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e8011-f755-4c50-95b1-6bd5e3408ea9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.441949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.442864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.443319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.443765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] PATH_INFO: `/v3/projects/68f59c81a2fc4a80ad6629cc1715c16b/users/1bd8c0e1f723420eaa5921a600f40957/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.455097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e8011-f755-4c50-95b1-6bd5e3408ea9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.460792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.461707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.462141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.462764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.463824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:52.464613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.466976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.471567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f59c81a2fc4a80ad6629cc1715c16b, user_id=1bd8c0e1f723420eaa5921a600f40957, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.476887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.491647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 920/1816] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:52.492587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.504536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094ec63-f751-4917-be50-80e2e1aab41f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.508636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 897/1817] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:52 2022] PUT /identity/v3/projects/68f59c81a2fc4a80ad6629cc1715c16b/users/1bd8c0e1f723420eaa5921a600f40957/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:52.513860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.603033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.603744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.604123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.604398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.605043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.607098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.621558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb99c4-6e2c-4651-a7b0-15d1863984d4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.628825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.629978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.630279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.630572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.631615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.633499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.645534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 898/1818] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:52.649933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.652920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 921/1819] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:52 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:52.663471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728525e-5ed7-4a74-8a94-840e7a0d693f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.664321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.745508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:52.746040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c81c9e-8d39-4741-b575-eb6db9e2f239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.746727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c81c9e-8d39-4741-b575-eb6db9e2f239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.747007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c81c9e-8d39-4741-b575-eb6db9e2f239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.747335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c81c9e-8d39-4741-b575-eb6db9e2f239 admin admin] PATH_INFO: `/v3/projects/68f59c81a2fc4a80ad6629cc1715c16b/users/1bd8c0e1f723420eaa5921a600f40957/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.749398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c81c9e-8d39-4741-b575-eb6db9e2f239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f59c81a2fc4a80ad6629cc1715c16b, user_id=1bd8c0e1f723420eaa5921a600f40957, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.752798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c81c9e-8d39-4741-b575-eb6db9e2f239 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.779219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3732f5-a00f-47e8-aad6-dcf510f59bb2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.792789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 899/1820] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:52 2022] PUT /identity/v3/projects/68f59c81a2fc4a80ad6629cc1715c16b/users/1bd8c0e1f723420eaa5921a600f40957/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:52.795022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.795367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.795628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.853490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:52.858204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] MFA Rules not processed for user `1bd8c0e1f723420eaa5921a600f40957`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:52.873363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:52.908672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.909414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.911622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.912170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.912838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.919407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.961733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 922/1821] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:52 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:52.963911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.978505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46125219-d49f-4ea9-80bd-9be65529ff00 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:52.985370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 900/1822] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:52.985454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:52.985454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:52.985454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:52.985662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:52.986344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:52.988376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:52.993704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:52.995164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:53.010802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 923/1823] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:53.012103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.035260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff9308-b683-46bf-b4bc-21635f0f917b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.052494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.053208 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.057168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.057845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.058689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.065685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.084621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ce904-bf4b-4d63-aee5-cc21e18e1635 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.089064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e634a-67bd-4c4d-ab1d-8497740e09b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.090260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e634a-67bd-4c4d-ab1d-8497740e09b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.090260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e634a-67bd-4c4d-ab1d-8497740e09b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.090782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e634a-67bd-4c4d-ab1d-8497740e09b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.104501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 924/1824] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:43:53.106109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.114889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e634a-67bd-4c4d-ab1d-8497740e09b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.116549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e634a-67bd-4c4d-ab1d-8497740e09b9 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.132699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 901/1825] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:53.132869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bba67-028d-4476-9dd0-1c0b2efe8215 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.149142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.149906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.150383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.150814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] PATH_INFO: `/v3/projects/41490ba9fd4046e4ba95f5fb93e07985/users/443d2e8c002746139548cfd49f0dd872/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.215759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41490ba9fd4046e4ba95f5fb93e07985, user_id=443d2e8c002746139548cfd49f0dd872, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.223008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.254584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 925/1826] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:53 2022] PUT /identity/v3/projects/41490ba9fd4046e4ba95f5fb93e07985/users/443d2e8c002746139548cfd49f0dd872/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:53.266828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ba34e-14c1-43c9-878a-a613b962f578 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.267534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ba34e-14c1-43c9-878a-a613b962f578 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.267534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ba34e-14c1-43c9-878a-a613b962f578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.314413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.317503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:53.372740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-de2ba34e-14c1-43c9-878a-a613b962f578 None None] MFA Rules not processed for user `443d2e8c002746139548cfd49f0dd872`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:53.373967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-de2ba34e-14c1-43c9-878a-a613b962f578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:53.462512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82c7bd-482d-4570-b387-37a4159a680e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.470090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.470784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.471070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.471546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.473452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:53.497960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 902/1827] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:53.503474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ba34e-14c1-43c9-878a-a613b962f578 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.551639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:53.558913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:53.576927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.578446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.589860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 926/1828] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:53.613367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ba34e-14c1-43c9-878a-a613b962f578 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.618003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.619081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.619081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.619422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.619906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.621880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.655287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 903/1829] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:53 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:53.659452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.671780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78ff7-d316-484f-9b34-131070171a73 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.738119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.739010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.739325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.739725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.740354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.742421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.770327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 927/1830] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:53 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:53.775742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.787907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5fefd-f98b-4415-8080-fff0b3ca34db admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.793608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.794315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.794679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.794999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.795692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.797324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.808968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 904/1831] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:53.813708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.824799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceafc0-c3e6-4353-a9d1-17d57e6c4537 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.831510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.832302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.832610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.832964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] PATH_INFO: `/v3/projects/6865819567a243c7ab35d85cfc3cb384/users/c2e1dc6c761349baafc083d84d21ba4e/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.854434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6865819567a243c7ab35d85cfc3cb384, user_id=c2e1dc6c761349baafc083d84d21ba4e, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.857431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.874560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 928/1832] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:53 2022] PUT /identity/v3/projects/6865819567a243c7ab35d85cfc3cb384/users/c2e1dc6c761349baafc083d84d21ba4e/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:53.880365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.891555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7586a-e7ce-499f-af4c-0ea2eb694e36 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.963232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b374f-1cf8-4131-9193-562121c65a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.964048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b374f-1cf8-4131-9193-562121c65a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.964341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b374f-1cf8-4131-9193-562121c65a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:53.964622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b374f-1cf8-4131-9193-562121c65a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:53.965321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b374f-1cf8-4131-9193-562121c65a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:53.967019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b374f-1cf8-4131-9193-562121c65a06 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:53.977349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 905/1833] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:53.982627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:53.993562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3192f-8261-4ea1-ac99-c226785d51be admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:53.998990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:53.999685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:53.999972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:54.000253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] PATH_INFO: `/v3/projects/6865819567a243c7ab35d85cfc3cb384/users/c2e1dc6c761349baafc083d84d21ba4e/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:54.002295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6865819567a243c7ab35d85cfc3cb384, user_id=c2e1dc6c761349baafc083d84d21ba4e, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:54.005034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:54.016634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 929/1834] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:53 2022] PUT /identity/v3/projects/6865819567a243c7ab35d85cfc3cb384/users/c2e1dc6c761349baafc083d84d21ba4e/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:54.027556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecf5a0-92fa-4d89-8233-f70e924dd406 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:54.027846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecf5a0-92fa-4d89-8233-f70e924dd406 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:54.028181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecf5a0-92fa-4d89-8233-f70e924dd406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:54.057953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-64ecf5a0-92fa-4d89-8233-f70e924dd406 None None] MFA Rules not processed for user `c2e1dc6c761349baafc083d84d21ba4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:54.059163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-64ecf5a0-92fa-4d89-8233-f70e924dd406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:54.172069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 906/1835] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:54.172799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:54.281659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fc098-345e-4a50-b278-0f46151ed15d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:54.286679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcad071-5ced-4e81-a500-d74d694ca3f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:54.287331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcad071-5ced-4e81-a500-d74d694ca3f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:54.287615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcad071-5ced-4e81-a500-d74d694ca3f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:54.287897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcad071-5ced-4e81-a500-d74d694ca3f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:54.300000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcad071-5ced-4e81-a500-d74d694ca3f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:54.301465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcad071-5ced-4e81-a500-d74d694ca3f9 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:54.313580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 930/1836] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:54.348179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecf5a0-92fa-4d89-8233-f70e924dd406 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:54.405876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:54.564722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecf5a0-92fa-4d89-8233-f70e924dd406 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:54.575598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:54.576447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:54.576823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:54.577194 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] PATH_INFO: `/v3/users/443d2e8c002746139548cfd49f0dd872` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:54.579274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] RBAC: Authorizing `identity:delete_user(user_id=443d2e8c002746139548cfd49f0dd872)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:54.582013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:54.728329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443d2e8c002746139548cfd49f0dd872'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:54.740242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443d2e8c002746139548cfd49f0dd872'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:54.754261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443d2e8c002746139548cfd49f0dd872'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:54.756499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443d2e8c002746139548cfd49f0dd872'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:54.757424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443d2e8c002746139548cfd49f0dd872'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:54.806249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:54.812538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:54.812942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:54.819993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 907/1837] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:54 2022] DELETE /identity/v3/users/443d2e8c002746139548cfd49f0dd872 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:54.853275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:54.855113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:54.942095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:54.986004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 931/1838] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:55.031185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:55.032678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:55.162990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4947b4-6e92-4e84-ae3b-e6c92ac5462c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:55.171533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:55.172279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:55.172667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:55.173044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:55.188324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:55.189305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:55.209932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 908/1839] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:56.422935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:56.425263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:56.464967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daad39-1783-4664-ab5e-5a15a36fe582 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:56.477953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:56.479127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:56.480270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:56.481317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] PATH_INFO: `/v3/users/2b6958bc50e94c1792191aa0adcc71cf` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:56.484437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6958bc50e94c1792191aa0adcc71cf)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:56.487123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:56.567399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6958bc50e94c1792191aa0adcc71cf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.568168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6958bc50e94c1792191aa0adcc71cf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.579743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6958bc50e94c1792191aa0adcc71cf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.580666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6958bc50e94c1792191aa0adcc71cf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.593755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b14e2089-1ffa-4537-9d71-8cf6e4efe55e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6958bc50e94c1792191aa0adcc71cf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.619629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:56.621388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 932/1840] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:56 2022] DELETE /identity/v3/users/2b6958bc50e94c1792191aa0adcc71cf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:56.622153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:56.627225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:56.627537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:56.628313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:56.640738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0a0cf-8237-4bf7-ae5c-db35be3656bd service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:56.656710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:56.657964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:56.723996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:56.741775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:56.743682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:56.744338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:56.744815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] PATH_INFO: `/v3/projects/41490ba9fd4046e4ba95f5fb93e07985` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:56.746547 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41490ba9fd4046e4ba95f5fb93e07985)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:56.748854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:56.751336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 933/1841] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:56.829523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41490ba9fd4046e4ba95f5fb93e07985', 'request_id': 'req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.834054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:56.835829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:56.837798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41490ba9fd4046e4ba95f5fb93e07985', 'request_id': 'req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.846399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41490ba9fd4046e4ba95f5fb93e07985', 'request_id': 'req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.864769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] The token cache is being invalidate because project 41490ba9fd4046e4ba95f5fb93e07985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:56.865285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.866440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:56.869549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 909/1842] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:56 2022] DELETE /identity/v3/projects/41490ba9fd4046e4ba95f5fb93e07985 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:56.874124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:56.876264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:57.000352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b58e-e7d3-42d7-accf-3cf555d0b541 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:57.006389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:57.007454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:57.007575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44fed-2512-4241-b6e6-e18eb35af062 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:57.008239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:57.009299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] PATH_INFO: `/v3/users/c2e1dc6c761349baafc083d84d21ba4e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:57.011731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e1dc6c761349baafc083d84d21ba4e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:57.014350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:57.038227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:57.108930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e1dc6c761349baafc083d84d21ba4e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:57.124939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e1dc6c761349baafc083d84d21ba4e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:57.142837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e1dc6c761349baafc083d84d21ba4e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:57.144054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e1dc6c761349baafc083d84d21ba4e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:57.144423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:57.144560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e1dc6c761349baafc083d84d21ba4e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:57.145080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:57.145372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:57.145639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:57.159544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:57.170091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:57.184385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:57.185040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 910/1843] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:56 2022] DELETE /identity/v3/users/c2e1dc6c761349baafc083d84d21ba4e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:57.186056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:57.210205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 934/1844] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:58.447281 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:58.449108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:58.556222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04d6a-3f3e-46b3-af20-ad9aeb491b6c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:58.562261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:58.563448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:58.572853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:58.573845 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:58.574307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:58.576341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] PATH_INFO: `/v3/projects/aaaed9f48803494a8e12c2cec1bc827a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:58.581947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaaed9f48803494a8e12c2cec1bc827a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:58.583975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:58.589163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5f115-9682-4852-a9c9-3b1259231ae6 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:58.597492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:58.598152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:58.598430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:58.598742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] PATH_INFO: `/v3/projects/6865819567a243c7ab35d85cfc3cb384` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:58.602604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6865819567a243c7ab35d85cfc3cb384)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:58.605629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:58.688495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaaed9f48803494a8e12c2cec1bc827a', 'request_id': 'req-55729948-3b6e-4b1c-af62-4aceb7ef282c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.697955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaaed9f48803494a8e12c2cec1bc827a', 'request_id': 'req-55729948-3b6e-4b1c-af62-4aceb7ef282c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.713620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaaed9f48803494a8e12c2cec1bc827a', 'request_id': 'req-55729948-3b6e-4b1c-af62-4aceb7ef282c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.730423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6865819567a243c7ab35d85cfc3cb384', 'request_id': 'req-3d7834d4-de2c-4052-8198-550a6989884e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.743916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6865819567a243c7ab35d85cfc3cb384', 'request_id': 'req-3d7834d4-de2c-4052-8198-550a6989884e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.750514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] The token cache is being invalidate because project aaaed9f48803494a8e12c2cec1bc827a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:58.751334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.752247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.754258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6865819567a243c7ab35d85cfc3cb384', 'request_id': 'req-3d7834d4-de2c-4052-8198-550a6989884e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.755456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 911/1845] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:58 2022] DELETE /identity/v3/projects/aaaed9f48803494a8e12c2cec1bc827a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:58.759618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:58.761802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:58.787741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] The token cache is being invalidate because project 6865819567a243c7ab35d85cfc3cb384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:58.788258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.789104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d7834d4-de2c-4052-8198-550a6989884e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:58.794690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 935/1846] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:58 2022] DELETE /identity/v3/projects/6865819567a243c7ab35d85cfc3cb384 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:58.800818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f23a-246c-4cf1-99ae-203d433e0f3d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:58.801109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f23a-246c-4cf1-99ae-203d433e0f3d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:58.801394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4f23a-246c-4cf1-99ae-203d433e0f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:58.832616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5aa4f23a-246c-4cf1-99ae-203d433e0f3d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:58.833853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5aa4f23a-246c-4cf1-99ae-203d433e0f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:58.873934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55729948-3b6e-4b1c-af62-4aceb7ef282c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:58.921806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:58.932395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 936/1847] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:58.934108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0ddd1-74fe-4170-8db2-bd2e10eb4012 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:58.934915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0ddd1-74fe-4170-8db2-bd2e10eb4012 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:58.934915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0ddd1-74fe-4170-8db2-bd2e10eb4012 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:58.937049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 937/1848] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:43:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:43:58.938206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353bf4a-908a-41df-adc2-710652eed4d0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:58.938555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353bf4a-908a-41df-adc2-710652eed4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:58.938903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353bf4a-908a-41df-adc2-710652eed4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:58.987432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6353bf4a-908a-41df-adc2-710652eed4d0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:58.988580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6353bf4a-908a-41df-adc2-710652eed4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:59.007107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 938/1849] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:59.007478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.008160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.008428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.008711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] PATH_INFO: `/v3/users/1bd8c0e1f723420eaa5921a600f40957` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.010032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd8c0e1f723420eaa5921a600f40957)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.012287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.014587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.014950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.015497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.068618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:59.070106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:59.134580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd8c0e1f723420eaa5921a600f40957'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.147045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd8c0e1f723420eaa5921a600f40957'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.153289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:59.163000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd8c0e1f723420eaa5921a600f40957'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.164016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd8c0e1f723420eaa5921a600f40957'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.165384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd8c0e1f723420eaa5921a600f40957'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.195090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 912/1850] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:58 2022] DELETE /identity/v3/users/1bd8c0e1f723420eaa5921a600f40957 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:43:59.196557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.198526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:59.239075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 939/1851] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:43:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:59.276834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.279849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:59.305365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048f864-8334-4261-a4e5-b7ac10caf8d7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.386394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabe81c-2671-43f9-b7ba-be978fd42e32 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.394869 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.396543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.396842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.397186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.397879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:59.398190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.400801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.429350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 913/1852] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:59.430563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.449064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b892def-ff2d-4b61-841a-94c92c322971 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.454072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.454241 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.454510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.454789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.455329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.456111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.456423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.456770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.457525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.459617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.476408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.478176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.507677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 940/1853] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:59.538590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 914/1854] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:59 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:59.543133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.557465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c38fa4-fdd7-49dc-94af-11bb685c48f0 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.688820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.690454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.690869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.691220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.691963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.699059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.743389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 941/1855] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:59 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:59.744509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.760190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62033c5-364a-4f4e-a0c1-bae5ce8b13a5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.766468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.767234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.768364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.769132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.770454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.773536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.795680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 915/1856] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:43:59.797205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.817417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a627ec-584c-418a-8c20-ae46671b8a23 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.826642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d7df8-9fbe-4afe-971f-2d01d26a4cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.827947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d7df8-9fbe-4afe-971f-2d01d26a4cad admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.828329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d7df8-9fbe-4afe-971f-2d01d26a4cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.829189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d7df8-9fbe-4afe-971f-2d01d26a4cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.830095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d7df8-9fbe-4afe-971f-2d01d26a4cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.833511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d7df8-9fbe-4afe-971f-2d01d26a4cad admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.855464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 942/1857] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:59.856737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.883033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb36a98-0faf-41f5-b5fd-88a025bf3337 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.890016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.890932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.891334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.892437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] PATH_INFO: `/v3/projects/a958f891533f4f4b891b7356fb3d5ce7/users/5f20104af7224a6383293eb9ccbf9948/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.908551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:59.928244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a958f891533f4f4b891b7356fb3d5ce7, user_id=5f20104af7224a6383293eb9ccbf9948, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.933358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.961657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 916/1858] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:59 2022] PUT /identity/v3/projects/a958f891533f4f4b891b7356fb3d5ce7/users/5f20104af7224a6383293eb9ccbf9948/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:59.976685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.977063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.977422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.018898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] MFA Rules not processed for user `5f20104af7224a6383293eb9ccbf9948`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:00.020903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.043536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:00.144282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.147152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 943/1859] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:43:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:00.249296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.251736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.261069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b6810-b101-4b66-8e00-110ff0cd1700 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.267159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.267918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.268270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.268534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.269237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.271450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.281655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90fe4d-7b08-4229-86a6-5a3b8ca9e889 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.288035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.288999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.289491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.289827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] PATH_INFO: `/v3/projects/68f59c81a2fc4a80ad6629cc1715c16b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.292030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f59c81a2fc4a80ad6629cc1715c16b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.294485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.308474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 917/1860] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:00.314246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.369790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f59c81a2fc4a80ad6629cc1715c16b', 'request_id': 'req-90cba216-987d-411b-98d7-024572dcb672'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.377228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f59c81a2fc4a80ad6629cc1715c16b', 'request_id': 'req-90cba216-987d-411b-98d7-024572dcb672'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.392531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f59c81a2fc4a80ad6629cc1715c16b', 'request_id': 'req-90cba216-987d-411b-98d7-024572dcb672'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.422125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154e5f-7f74-4bfc-af6a-364d6ab75ba4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.431774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] The token cache is being invalidate because project 68f59c81a2fc4a80ad6629cc1715c16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:00.432112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.432878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-90cba216-987d-411b-98d7-024572dcb672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.435171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 944/1861] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:00 2022] DELETE /identity/v3/projects/68f59c81a2fc4a80ad6629cc1715c16b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:00.440196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d10a0e-287a-4066-aa86-1d5195c4b7c5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.440556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d10a0e-287a-4066-aa86-1d5195c4b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.440949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d10a0e-287a-4066-aa86-1d5195c4b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.462323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-41d10a0e-287a-4066-aa86-1d5195c4b7c5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:00.463799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41d10a0e-287a-4066-aa86-1d5195c4b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.501845 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.502585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.502941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.503352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.504083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.506289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.554481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 918/1862] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:00.558978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.560847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.580058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b26c3-7b70-499d-afa9-075cc17d252b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.585552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.586463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.586749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.587027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.588190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.589907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.591803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 945/1863] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:00.599892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3677a-456b-4358-8d79-1e90951e02a0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.600554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3677a-456b-4358-8d79-1e90951e02a0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.601347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3677a-456b-4358-8d79-1e90951e02a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.611710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 919/1864] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:00.612948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.621083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:00.635145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ff7c6-194c-4ddd-8afa-ee0962666cc3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.645000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.646096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.646468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.646819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] PATH_INFO: `/v3/projects/9f8f8b66efda4ed99a36a0ae751f4e9e/users/74b96713e6fe4b248e670aa5cdd5d35c/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.647170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-28e3677a-456b-4358-8d79-1e90951e02a0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:00.653355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-28e3677a-456b-4358-8d79-1e90951e02a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.676302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 946/1865] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:00.682709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3677a-456b-4358-8d79-1e90951e02a0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.685124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8f8b66efda4ed99a36a0ae751f4e9e, user_id=74b96713e6fe4b248e670aa5cdd5d35c, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.692798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.706901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3677a-456b-4358-8d79-1e90951e02a0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.723578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.724225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.724507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.726570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.728397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:00.728961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.730438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.749167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 920/1866] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:00 2022] PUT /identity/v3/projects/9f8f8b66efda4ed99a36a0ae751f4e9e/users/74b96713e6fe4b248e670aa5cdd5d35c/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:00.750213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.759234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 947/1867] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:44:00.761380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.789524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03da8-a059-4b16-b44f-eeef683a767e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.842288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:00.909901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de69d4-1993-47ef-8372-78fb9504749a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.919134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.919976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.920399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.920778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.921507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.924324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.930577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.931300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.931779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.932068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.932728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.934678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.946132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 921/1868] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:00.947430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.965596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ef6b2-1cda-41b9-9863-6d67f296db7f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.975693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278ba1c-3003-41c9-8d08-350266e8a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.978236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278ba1c-3003-41c9-8d08-350266e8a8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.979224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278ba1c-3003-41c9-8d08-350266e8a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.979763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278ba1c-3003-41c9-8d08-350266e8a8aa admin admin] PATH_INFO: `/v3/projects/9f8f8b66efda4ed99a36a0ae751f4e9e/users/74b96713e6fe4b248e670aa5cdd5d35c/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.982886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278ba1c-3003-41c9-8d08-350266e8a8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8f8b66efda4ed99a36a0ae751f4e9e, user_id=74b96713e6fe4b248e670aa5cdd5d35c, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.986690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278ba1c-3003-41c9-8d08-350266e8a8aa admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.001389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 948/1869] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:01.006012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:01.034839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 922/1870] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:00 2022] PUT /identity/v3/projects/9f8f8b66efda4ed99a36a0ae751f4e9e/users/74b96713e6fe4b248e670aa5cdd5d35c/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:01.040206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.040500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.040787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.131097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] MFA Rules not processed for user `74b96713e6fe4b248e670aa5cdd5d35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:01.133054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:01.172345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e060e-f503-4307-a3eb-de0a5023859f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.177879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.178637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.178975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.179352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.180058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.182131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.215299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 949/1871] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:01 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:01.217511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:01.230187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d7258-823b-449b-9b0c-eb2ef40f8e90 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.235658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.236339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.236666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.236946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.237772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.239496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.251112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 950/1872] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:01.259329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:01.298166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e29fd-e5c6-48f3-9aad-e8e933a75785 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.309957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.310603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.310910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.311175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.312234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.317976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.350411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 951/1873] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:01.351574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:01.378231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9cdbc-a312-4876-810f-9a841a9a6b7d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.383374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 923/1874] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:01.394945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:01.397162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:01.400742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9b6c9-92a7-49c1-a76f-fd16d689ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.401944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9b6c9-92a7-49c1-a76f-fd16d689ee34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.402247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9b6c9-92a7-49c1-a76f-fd16d689ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.402574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9b6c9-92a7-49c1-a76f-fd16d689ee34 admin admin] PATH_INFO: `/v3/projects/6f99dbcc7baf41dd82bc7bcbb05985b7/users/d225516f41a5473cb19bf83579c8c1b3/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.450170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9b6c9-92a7-49c1-a76f-fd16d689ee34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f99dbcc7baf41dd82bc7bcbb05985b7, user_id=d225516f41a5473cb19bf83579c8c1b3, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.460290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9b6c9-92a7-49c1-a76f-fd16d689ee34 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.504197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 952/1875] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:01 2022] PUT /identity/v3/projects/6f99dbcc7baf41dd82bc7bcbb05985b7/users/d225516f41a5473cb19bf83579c8c1b3/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:01.512665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.513226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.513653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.537974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:01.546470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac10d-6357-40b3-8e33-a20704c935ad None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.573137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] MFA Rules not processed for user `d225516f41a5473cb19bf83579c8c1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:01.574392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:01.622700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.622700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.623040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.623274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.628602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:01.665962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 953/1876] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:01.677173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:01.681849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:01.706381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.706733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.708208 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc271-2b50-4782-8af9-a3a4e8a2455d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.712682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781d284-a2d4-4503-a937-ddadc036b157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.713550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d284-a2d4-4503-a937-ddadc036b157 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.713851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d284-a2d4-4503-a937-ddadc036b157 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.714160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781d284-a2d4-4503-a937-ddadc036b157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.726516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 924/1877] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:01.726944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781d284-a2d4-4503-a937-ddadc036b157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.730285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781d284-a2d4-4503-a937-ddadc036b157 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.744095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 954/1878] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:01.817002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:01.917198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd2ce0-c863-4f9c-b0f7-a2501c3d971f service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.922507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.923274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.923559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.923912 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] PATH_INFO: `/v3/users/5f20104af7224a6383293eb9ccbf9948` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.925224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f20104af7224a6383293eb9ccbf9948)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.927293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.981207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f20104af7224a6383293eb9ccbf9948'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:01.987652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:01.989962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f20104af7224a6383293eb9ccbf9948'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:01.998513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f20104af7224a6383293eb9ccbf9948'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:01.999267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f20104af7224a6383293eb9ccbf9948'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:01.999982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f20104af7224a6383293eb9ccbf9948'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:02.026361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 925/1879] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:01 2022] DELETE /identity/v3/users/5f20104af7224a6383293eb9ccbf9948 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:02.026465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:02.026465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:02.026465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:02.047902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:02.049762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:02.123710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 955/1880] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:02.134179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:02.135786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:02.197050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8e8fa-5feb-48ba-a75d-1af9deaeebc2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:02.201434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:02.202057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:02.202335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:02.202611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:02.213193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:02.214643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:02.225172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 926/1881] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:02.905248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:02.906838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:02.924397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025dda7-5cea-4d13-a77f-ddda3da2c8cb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:02.932128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:02.932884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:02.933247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:02.933602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] PATH_INFO: `/v3/projects/a958f891533f4f4b891b7356fb3d5ce7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:02.935167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a958f891533f4f4b891b7356fb3d5ce7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:02.939375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:03.009250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a958f891533f4f4b891b7356fb3d5ce7', 'request_id': 'req-30d135fb-3e00-4648-930e-d01a611fab16'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.016981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a958f891533f4f4b891b7356fb3d5ce7', 'request_id': 'req-30d135fb-3e00-4648-930e-d01a611fab16'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.031778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a958f891533f4f4b891b7356fb3d5ce7', 'request_id': 'req-30d135fb-3e00-4648-930e-d01a611fab16'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.048169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] The token cache is being invalidate because project a958f891533f4f4b891b7356fb3d5ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:03.048469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.049114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.050633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 956/1882] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:02 2022] DELETE /identity/v3/projects/a958f891533f4f4b891b7356fb3d5ce7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:03.055110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:03.065770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86281-037a-45f1-a7c6-927439b261e2 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:03.140080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:03.141029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:03.141512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:03.142006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] PATH_INFO: `/v3/users/74b96713e6fe4b248e670aa5cdd5d35c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:03.143543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b96713e6fe4b248e670aa5cdd5d35c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:03.145847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:03.209958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b96713e6fe4b248e670aa5cdd5d35c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.229170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b96713e6fe4b248e670aa5cdd5d35c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.242313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b96713e6fe4b248e670aa5cdd5d35c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.243100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b96713e6fe4b248e670aa5cdd5d35c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.244848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8323ae32-f771-456b-9358-ea906ca25647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b96713e6fe4b248e670aa5cdd5d35c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.262883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:03.274680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 927/1883] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:03 2022] DELETE /identity/v3/users/74b96713e6fe4b248e670aa5cdd5d35c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:03.581491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:03.583427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:03.599450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d135fb-3e00-4648-930e-d01a611fab16 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:03.648537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:03.663285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:03.663902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:03.664194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:03.664476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] PATH_INFO: `/v3/projects/9f8f8b66efda4ed99a36a0ae751f4e9e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:03.665852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8f8b66efda4ed99a36a0ae751f4e9e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:03.667995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:03.746226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8f8b66efda4ed99a36a0ae751f4e9e', 'request_id': 'req-b07cb77c-f218-4174-bbf7-f56efbb4562d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.753881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8f8b66efda4ed99a36a0ae751f4e9e', 'request_id': 'req-b07cb77c-f218-4174-bbf7-f56efbb4562d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.761833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8f8b66efda4ed99a36a0ae751f4e9e', 'request_id': 'req-b07cb77c-f218-4174-bbf7-f56efbb4562d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.790206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] The token cache is being invalidate because project 9f8f8b66efda4ed99a36a0ae751f4e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:03.791528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.792441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b07cb77c-f218-4174-bbf7-f56efbb4562d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:03.799507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33390b3-a927-437d-a290-ebeebe11cd42 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:03.799801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33390b3-a927-437d-a290-ebeebe11cd42 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:03.800268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33390b3-a927-437d-a290-ebeebe11cd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:03.803278 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 957/1884] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:03 2022] DELETE /identity/v3/projects/9f8f8b66efda4ed99a36a0ae751f4e9e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:03.827062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a33390b3-a927-437d-a290-ebeebe11cd42 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:03.828724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a33390b3-a927-437d-a290-ebeebe11cd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:03.918298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 928/1885] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:03.933129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17259de-28d8-40ea-b8f1-fb00c39c8682 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:03.933652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17259de-28d8-40ea-b8f1-fb00c39c8682 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:03.933730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17259de-28d8-40ea-b8f1-fb00c39c8682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:03.954805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c17259de-28d8-40ea-b8f1-fb00c39c8682 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:03.956186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c17259de-28d8-40ea-b8f1-fb00c39c8682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:03.967230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 958/1886] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:03.969344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33390b3-a927-437d-a290-ebeebe11cd42 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:03.980397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33390b3-a927-437d-a290-ebeebe11cd42 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:03.985778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:03.986754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:03.986754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:03.987044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:03.987798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:03.988129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:03.989767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.003378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 929/1887] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:04.007016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17259de-28d8-40ea-b8f1-fb00c39c8682 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.017131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17259de-28d8-40ea-b8f1-fb00c39c8682 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.029892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.030526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.030814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.031090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.031768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.033701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.076168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 959/1888] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:04 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:04.080517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.092680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd8367-e2eb-4008-ac08-9686b1bd1a92 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.162544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.163881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.164207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.164557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.165689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.167903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.199617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 930/1889] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:04 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:04.205445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.234274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045bcef-dfa3-4d98-aaba-51e8110d0428 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.239477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.240256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.240608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.240951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.241680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.243536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.259396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 960/1890] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:04.268462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.279593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569bc91-1462-42c9-971e-3137c14418f5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.287171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c4dd7-2b4b-40ae-89c9-d712456b778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.287835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c4dd7-2b4b-40ae-89c9-d712456b778b admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.288125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c4dd7-2b4b-40ae-89c9-d712456b778b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.288405 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c4dd7-2b4b-40ae-89c9-d712456b778b admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.289392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c4dd7-2b4b-40ae-89c9-d712456b778b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.291740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c4dd7-2b4b-40ae-89c9-d712456b778b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.305428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 931/1891] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:04.309771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.327260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e506592-6251-4110-8463-427a9d919128 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.333930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.334822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.335183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.335574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] PATH_INFO: `/v3/projects/9888d2b17cfa4a53940c89e0a4964127/users/38a6a563ffb64118be7a7f8ea0eec755/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.359874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9888d2b17cfa4a53940c89e0a4964127, user_id=38a6a563ffb64118be7a7f8ea0eec755, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.363842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.392780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 961/1892] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:04 2022] PUT /identity/v3/projects/9888d2b17cfa4a53940c89e0a4964127/users/38a6a563ffb64118be7a7f8ea0eec755/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:04.394724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf76bc-e861-4208-b930-dbbef875b1af None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.395418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf76bc-e861-4208-b930-dbbef875b1af None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.395880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf76bc-e861-4208-b930-dbbef875b1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.427829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-37cf76bc-e861-4208-b930-dbbef875b1af None None] MFA Rules not processed for user `38a6a563ffb64118be7a7f8ea0eec755`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:04.429222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-37cf76bc-e861-4208-b930-dbbef875b1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:04.505280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:04.542853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 932/1893] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:04.543760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.681751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30676f28-ac29-4f27-a551-31dc4b6089b7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.688505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.688973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.690968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.690968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.690968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.694717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.739530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:04.757007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 962/1894] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:04 2022] POST /identity/v3/projects => generated 387 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:04.762364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf76bc-e861-4208-b930-dbbef875b1af None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.783561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf76bc-e861-4208-b930-dbbef875b1af None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.885968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.886850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.887225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.888346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.889532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.892190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:04.954311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 933/1895] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:04 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:04.967457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:04.971007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec99c-7a1a-4dbc-baaf-4bfeeee0dd39 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:04.980287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:04.981256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:04.981671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:04.981999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:04.983058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:04.995724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.007135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.010082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:05.029134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 963/1896] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:05.029134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.032020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090fb98-6762-4c4b-9479-3cb3cb628842 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.039951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.039951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.039951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.039951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] PATH_INFO: `/v3/users/d225516f41a5473cb19bf83579c8c1b3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.043250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d225516f41a5473cb19bf83579c8c1b3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.045910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.052253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b5a52-d8f2-4660-84aa-16d65d79faf3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.058186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.059176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.059681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.060153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.061076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.063446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.084751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 964/1897] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:05.089797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.112256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674f85-53b6-444a-bcea-ad8934a8cb14 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.134468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa087a4b-3600-4b60-acbf-c6b04a08979a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.135183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa087a4b-3600-4b60-acbf-c6b04a08979a admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.135501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa087a4b-3600-4b60-acbf-c6b04a08979a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.135781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa087a4b-3600-4b60-acbf-c6b04a08979a admin admin] PATH_INFO: `/v3/projects/2dd31875cb194c91a2f734f7d4e29108/users/c20676affe6f457a865c8a17442a6e55/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.137730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd225516f41a5473cb19bf83579c8c1b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:05.150451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd225516f41a5473cb19bf83579c8c1b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:05.155507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa087a4b-3600-4b60-acbf-c6b04a08979a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd31875cb194c91a2f734f7d4e29108, user_id=c20676affe6f457a865c8a17442a6e55, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.158400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa087a4b-3600-4b60-acbf-c6b04a08979a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.166735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd225516f41a5473cb19bf83579c8c1b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:05.167934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd225516f41a5473cb19bf83579c8c1b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:05.169360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6cada841-5922-43fd-99ea-426f551a1d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd225516f41a5473cb19bf83579c8c1b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:05.179293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 965/1898] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:05 2022] PUT /identity/v3/projects/2dd31875cb194c91a2f734f7d4e29108/users/c20676affe6f457a865c8a17442a6e55/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:05.190381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.190974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.191388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.233015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 934/1899] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:05 2022] DELETE /identity/v3/users/d225516f41a5473cb19bf83579c8c1b3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:05.238070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.238070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.238924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.240857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] MFA Rules not processed for user `c20676affe6f457a865c8a17442a6e55`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:05.242508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:05.262025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:05.263694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:05.332326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:05.348615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 966/1900] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:05.350653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.352065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:05.356899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 935/1901] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:05.366108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.367004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f716e21-241c-4291-af71-1c01266a3727 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.367594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:05.372459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.373105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.373393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.373668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.374361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.376396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.414070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 967/1902] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:05 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:05.419291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.446593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30a625-e5e6-4f5e-a9a8-1dad9a33b7a6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.497335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf731a6-082c-46a7-858c-402831dd61a7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.504201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.505596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.505923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.506201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.507007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.509712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.535502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.536135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.536424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.536846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.550161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.551115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 968/1903] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:05 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:05.551489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.552701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.565809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5cb0-51a1-4368-a711-6bee0fef009a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.574020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.574940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.575398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.575823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.576725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.578093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 936/1904] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:05.579508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.587870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:05.597872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 969/1905] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:05.598810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.610569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77d7c-209d-445b-b8ae-97187ac1baa6 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.619424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.620185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.620491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.620730 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] PATH_INFO: `/v3/projects/2c0c28a3c5704898b43a1dd7ef68a5a2/users/18dd6751b0fa4d18a06f9d5bbe0f0746/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.662101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0c28a3c5704898b43a1dd7ef68a5a2, user_id=18dd6751b0fa4d18a06f9d5bbe0f0746, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.666761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.691322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 937/1906] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:05 2022] PUT /identity/v3/projects/2c0c28a3c5704898b43a1dd7ef68a5a2/users/18dd6751b0fa4d18a06f9d5bbe0f0746/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:05.695150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.713945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4ffc0-a25d-4f59-abaf-0a5c25f142c4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.813066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35520918-6285-46b6-ac8f-926f3a01b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.814441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35520918-6285-46b6-ac8f-926f3a01b89e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.814733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35520918-6285-46b6-ac8f-926f3a01b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.815010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35520918-6285-46b6-ac8f-926f3a01b89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.815885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35520918-6285-46b6-ac8f-926f3a01b89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.817649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35520918-6285-46b6-ac8f-926f3a01b89e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.837602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 970/1907] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:05.842685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:05.858679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17e3b-6943-4e5b-93c1-4fd5295eb51e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:05.865053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:05.865859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.866074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.866408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] PATH_INFO: `/v3/projects/2c0c28a3c5704898b43a1dd7ef68a5a2/users/18dd6751b0fa4d18a06f9d5bbe0f0746/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.868521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0c28a3c5704898b43a1dd7ef68a5a2, user_id=18dd6751b0fa4d18a06f9d5bbe0f0746, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:05.871828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:05.884178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 938/1908] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:05 2022] PUT /identity/v3/projects/2c0c28a3c5704898b43a1dd7ef68a5a2/users/18dd6751b0fa4d18a06f9d5bbe0f0746/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:05.901249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:05.902541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:05.902850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:05.964941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] MFA Rules not processed for user `18dd6751b0fa4d18a06f9d5bbe0f0746`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:05.966357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:06.169933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 971/1909] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:06.175591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:06.179245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:06.268977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:06.270413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:06.311081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1bb3a-afc0-4a7c-9740-e21d9b72e555 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:06.322520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:06.323285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:06.323575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:06.323853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:06.343124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:06.346647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:06.360293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 939/1910] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:06.364133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:06.366096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:06.401170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:06.422957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90ec75-aebf-4442-9528-8e7920f82d10 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:06.430460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:06.431435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:06.431724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:06.432003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] PATH_INFO: `/v3/projects/6f99dbcc7baf41dd82bc7bcbb05985b7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:06.433249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f99dbcc7baf41dd82bc7bcbb05985b7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:06.435246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:06.487491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:06.501709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8cee-662d-4da6-bfac-09fdc665ad22 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:06.510650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:06.511393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:06.511715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:06.511999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] PATH_INFO: `/v3/users/b3a4dc2eb6634a9db57083a7ac819c47` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:06.513261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a4dc2eb6634a9db57083a7ac819c47)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:06.516524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:06.549166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f99dbcc7baf41dd82bc7bcbb05985b7', 'request_id': 'req-543f86e2-b791-4eae-b539-cef69c42f40f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.562991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f99dbcc7baf41dd82bc7bcbb05985b7', 'request_id': 'req-543f86e2-b791-4eae-b539-cef69c42f40f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.590665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f99dbcc7baf41dd82bc7bcbb05985b7', 'request_id': 'req-543f86e2-b791-4eae-b539-cef69c42f40f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.625116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] The token cache is being invalidate because project 6f99dbcc7baf41dd82bc7bcbb05985b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:06.625777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.629159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-543f86e2-b791-4eae-b539-cef69c42f40f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.637643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 972/1911] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:06 2022] DELETE /identity/v3/projects/6f99dbcc7baf41dd82bc7bcbb05985b7 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:06.641215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527559d-99b2-4994-9df4-619434843ac8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:06.642470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527559d-99b2-4994-9df4-619434843ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:06.642843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527559d-99b2-4994-9df4-619434843ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:06.680815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a4dc2eb6634a9db57083a7ac819c47'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.696695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7527559d-99b2-4994-9df4-619434843ac8 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:06.698249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7527559d-99b2-4994-9df4-619434843ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:06.719016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a4dc2eb6634a9db57083a7ac819c47'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.771557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a4dc2eb6634a9db57083a7ac819c47'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.772563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a4dc2eb6634a9db57083a7ac819c47'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.773942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-81db9902-5a61-49c4-b252-203fc7d84621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a4dc2eb6634a9db57083a7ac819c47'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:06.863942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 940/1912] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:06 2022] DELETE /identity/v3/users/b3a4dc2eb6634a9db57083a7ac819c47 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:06.874177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:06.874997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:06.876982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:06.886615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 973/1913] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:06.891533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793994bb-a2d6-4993-a768-7616ad4cbc03 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:06.892061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793994bb-a2d6-4993-a768-7616ad4cbc03 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:06.892483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793994bb-a2d6-4993-a768-7616ad4cbc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:06.945015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:06.948012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:06.948518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-793994bb-a2d6-4993-a768-7616ad4cbc03 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:06.952004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-793994bb-a2d6-4993-a768-7616ad4cbc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:07.107974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 941/1914] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:07.123321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 974/1915] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:07.124122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.126368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793994bb-a2d6-4993-a768-7616ad4cbc03 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.136312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:07.143402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793994bb-a2d6-4993-a768-7616ad4cbc03 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.156666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.157433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.157720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.158221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.158823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:07.159066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.160771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.175332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 975/1916] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:07.179919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.193564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc000f-946d-4918-8765-69078d0d5bd6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.201317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.202795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.203090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.203401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.204166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.206653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.220738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:07.257866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26c52a-c612-4d8b-bc31-97e173fc57fb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.269600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.270364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.270729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.271007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.281077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 976/1917] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:07 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:07.283540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.304363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.305876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.318759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:07.336558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 942/1918] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:07.379419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8ccf5-43d8-4f11-9fbb-c06c42cf672c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.386374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.387014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.387349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.387627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.388276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.390456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.424157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 977/1919] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:07 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:07.426718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.435775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.438210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:07.440232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76341f-9959-4c7b-b5d1-f36666792261 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.445727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.446537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.446818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.447162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.447876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.449813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.465520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 943/1920] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:07.469954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.488578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff82700-a9e8-4885-824b-839c2bffc74f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.500804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.502644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.502920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.505640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.506338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.509770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.518173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa9cf4-3b2f-4465-8dc6-51358f221bcb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.527978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.528926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.529392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.529832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.529966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 944/1921] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:07.531950 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:07.535998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.560448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123370d-de52-4656-bcc3-1274286c329a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:07.577011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725a501-c5f3-4c96-b8b2-ffe3559b0e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:07.577728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725a501-c5f3-4c96-b8b2-ffe3559b0e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.578024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725a501-c5f3-4c96-b8b2-ffe3559b0e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.578307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725a501-c5f3-4c96-b8b2-ffe3559b0e5d admin admin] PATH_INFO: `/v3/projects/09db79d382e149acbaac327f95ae252d/users/b2d41acfd6474e78929fd674a59233a9/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.616085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725a501-c5f3-4c96-b8b2-ffe3559b0e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=09db79d382e149acbaac327f95ae252d, user_id=b2d41acfd6474e78929fd674a59233a9, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.620263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725a501-c5f3-4c96-b8b2-ffe3559b0e5d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.647432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:07.649067 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:07.675706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 945/1922] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:07 2022] PUT /identity/v3/projects/09db79d382e149acbaac327f95ae252d/users/b2d41acfd6474e78929fd674a59233a9/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:07.689182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:07.689457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:07.689802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:07.690636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 978/1923] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:07.783256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] MFA Rules not processed for user `b2d41acfd6474e78929fd674a59233a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:07.784889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:07.946977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.950027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:07.971168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 946/1924] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:07.980758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:07.982745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:07.983038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e622f-e552-4cc6-a26a-ea9396a2a615 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:08.101144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:08.101769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:08.102059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:08.102337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] PATH_INFO: `/v3/projects/8cd7ef6de68d49b19fdf31e38b49d73b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:08.103121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1305e-d299-4257-b842-b8dbaf46197a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:08.103867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd7ef6de68d49b19fdf31e38b49d73b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:08.114717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:08.117628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:08.125556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:08.125978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:08.126264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:08.126534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:08.167646 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:08.169125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:08.200824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 947/1925] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:08.230526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd7ef6de68d49b19fdf31e38b49d73b', 'request_id': 'req-9a005c50-1915-4b78-8c74-9057ffad5d23'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.241048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd7ef6de68d49b19fdf31e38b49d73b', 'request_id': 'req-9a005c50-1915-4b78-8c74-9057ffad5d23'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.249850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd7ef6de68d49b19fdf31e38b49d73b', 'request_id': 'req-9a005c50-1915-4b78-8c74-9057ffad5d23'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.271742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] The token cache is being invalidate because project 8cd7ef6de68d49b19fdf31e38b49d73b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:08.272039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.272762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.278609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 979/1926] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:07 2022] DELETE /identity/v3/projects/8cd7ef6de68d49b19fdf31e38b49d73b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:08.281014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:08.300562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f3519-a256-42c5-842d-429602b0fbb7 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:08.358007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:08.419080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:08.419771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:08.420058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:08.420352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] PATH_INFO: `/v3/users/4c3e79238cb24c0490db1d897baa6c7b` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:08.421504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3e79238cb24c0490db1d897baa6c7b)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:08.423558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:08.487475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3e79238cb24c0490db1d897baa6c7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.495756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3e79238cb24c0490db1d897baa6c7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.506378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3e79238cb24c0490db1d897baa6c7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.507515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3e79238cb24c0490db1d897baa6c7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.508297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d29637cd-3d5b-43f2-aa49-95a18026d9ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3e79238cb24c0490db1d897baa6c7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:08.563675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 948/1927] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:08 2022] DELETE /identity/v3/users/4c3e79238cb24c0490db1d897baa6c7b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:08.950668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:08.952371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:08.970214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005c50-1915-4b78-8c74-9057ffad5d23 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.045939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.046655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.047097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.047923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] PATH_INFO: `/v3/projects/86f01233e6624b45ac1306380839b2e5` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.048669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f01233e6624b45ac1306380839b2e5)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.050781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.114776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f01233e6624b45ac1306380839b2e5', 'request_id': 'req-cd0aee3a-3984-4238-b6ca-41792693584b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:09.123741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f01233e6624b45ac1306380839b2e5', 'request_id': 'req-cd0aee3a-3984-4238-b6ca-41792693584b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:09.134102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f01233e6624b45ac1306380839b2e5', 'request_id': 'req-cd0aee3a-3984-4238-b6ca-41792693584b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:09.155008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] The token cache is being invalidate because project 86f01233e6624b45ac1306380839b2e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:09.155583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:09.156305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd0aee3a-3984-4238-b6ca-41792693584b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:09.161489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 980/1928] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:08 2022] DELETE /identity/v3/projects/86f01233e6624b45ac1306380839b2e5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:09.163847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1f5f-2ee9-4d9d-bbc8-9762e749f249 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.163847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1f5f-2ee9-4d9d-bbc8-9762e749f249 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.164085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1f5f-2ee9-4d9d-bbc8-9762e749f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.187186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3afc1f5f-2ee9-4d9d-bbc8-9762e749f249 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:09.188420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3afc1f5f-2ee9-4d9d-bbc8-9762e749f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:09.274655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 949/1929] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:09.282107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e6f7-976e-4298-8b74-9edd38847c55 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.282622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e6f7-976e-4298-8b74-9edd38847c55 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.282810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e6f7-976e-4298-8b74-9edd38847c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.314967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ac57e6f7-976e-4298-8b74-9edd38847c55 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:09.316793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ac57e6f7-976e-4298-8b74-9edd38847c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:09.335069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 981/1930] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:09.340689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1f5f-2ee9-4d9d-bbc8-9762e749f249 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:09.355303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1f5f-2ee9-4d9d-bbc8-9762e749f249 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.368831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.369749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.370132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.370507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.371378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:09.371948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.373817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.391649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 950/1931] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:09.397419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57e6f7-976e-4298-8b74-9edd38847c55 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:09.409275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57e6f7-976e-4298-8b74-9edd38847c55 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.415644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.416385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.416854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.417236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.418405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.421222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.444546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:09.469597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 982/1932] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:09 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:09.480877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:09.498310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc75cd-8f48-4fb9-8d71-5027d3ba9ca2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.553018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:09.610975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.611847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.612789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.613729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.617492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.626139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.675714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 951/1933] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:09 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:09.677077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:09.692245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bf487-c80a-4e2f-ad8e-0f9ef9ef1202 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.699170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.700751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.701225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.701698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.702605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.705083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.722993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 983/1934] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:09.724344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:09.737188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa070d08-224f-4925-8674-83ab61c44cfc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.746102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf65c7-2550-41f1-9c31-6f527d31746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.747089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf65c7-2550-41f1-9c31-6f527d31746e admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.747829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf65c7-2550-41f1-9c31-6f527d31746e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.748211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf65c7-2550-41f1-9c31-6f527d31746e admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.749054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf65c7-2550-41f1-9c31-6f527d31746e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.751042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf65c7-2550-41f1-9c31-6f527d31746e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.766526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 952/1935] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:09.767844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:09.779630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9d742-103e-4ec8-bf54-5930ae907a12 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.786015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.787040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.787152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.787637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] PATH_INFO: `/v3/projects/4d228d0afa7c412fb1c0088e008edac3/users/4305d5ed53d64205a5fa1d0588d80e13/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.804998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d228d0afa7c412fb1c0088e008edac3, user_id=4305d5ed53d64205a5fa1d0588d80e13, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.808211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.830960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 984/1936] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:09 2022] PUT /identity/v3/projects/4d228d0afa7c412fb1c0088e008edac3/users/4305d5ed53d64205a5fa1d0588d80e13/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:09.843413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4356e-f2fb-4d1a-88d0-8db67737bd23 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.844146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4356e-f2fb-4d1a-88d0-8db67737bd23 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.844481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4356e-f2fb-4d1a-88d0-8db67737bd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.880050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c7a4356e-f2fb-4d1a-88d0-8db67737bd23 None None] MFA Rules not processed for user `4305d5ed53d64205a5fa1d0588d80e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:09.881715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c7a4356e-f2fb-4d1a-88d0-8db67737bd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:10.026807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 953/1937] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:10.035249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:10.145698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7c4e-02d4-4d80-932c-79fc8e9de907 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:10.151222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:10.151856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.152144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.153117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:10.153747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:10.155786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:10.187238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 985/1938] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:10 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:10.191377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4356e-f2fb-4d1a-88d0-8db67737bd23 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:10.204069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4356e-f2fb-4d1a-88d0-8db67737bd23 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:10.280686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:10.281885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.282355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.282641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:10.283540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:10.285787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:10.312541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 954/1939] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:10 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:10.316179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:10.329371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4876cf-0546-4311-8354-2ac4fc959bd5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:10.334769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:10.335499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.335704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.335983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:10.336654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:10.338735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:10.351295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 986/1940] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:10.356517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:10.382155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9981e48-17b5-49b1-b79c-dcfda270e7a6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:10.392406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:10.396702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.397177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.398198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] PATH_INFO: `/v3/projects/5a2a36368bad459f869fb2898aa33510/users/7d268785bf4d441d95724e249f7b06a9/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:10.446867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2a36368bad459f869fb2898aa33510, user_id=7d268785bf4d441d95724e249f7b06a9, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:10.450214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:10.474124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 955/1941] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:10 2022] PUT /identity/v3/projects/5a2a36368bad459f869fb2898aa33510/users/7d268785bf4d441d95724e249f7b06a9/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:10.478413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:10.496646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dbeb5-6167-4dae-a8e0-16545e4f3572 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:10.610778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac357a-bc37-42b0-a579-dbc08e457e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:10.611520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac357a-bc37-42b0-a579-dbc08e457e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.611725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac357a-bc37-42b0-a579-dbc08e457e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.611994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac357a-bc37-42b0-a579-dbc08e457e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:10.613412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac357a-bc37-42b0-a579-dbc08e457e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:10.614666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac357a-bc37-42b0-a579-dbc08e457e91 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:10.626237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 987/1942] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:10.630232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:10.638611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:10.643360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f666d7-2f0e-49cd-8d6f-5b9e89459238 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:10.649958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:10.651173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.655584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.655978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] PATH_INFO: `/v3/projects/5a2a36368bad459f869fb2898aa33510/users/7d268785bf4d441d95724e249f7b06a9/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:10.658451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2a36368bad459f869fb2898aa33510, user_id=7d268785bf4d441d95724e249f7b06a9, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:10.661433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:10.679259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 956/1943] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:10 2022] PUT /identity/v3/projects/5a2a36368bad459f869fb2898aa33510/users/7d268785bf4d441d95724e249f7b06a9/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:10.686317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe582454-6947-482a-b911-aa1c06534da7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.686983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe582454-6947-482a-b911-aa1c06534da7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.687318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe582454-6947-482a-b911-aa1c06534da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:10.725043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fe582454-6947-482a-b911-aa1c06534da7 None None] MFA Rules not processed for user `7d268785bf4d441d95724e249f7b06a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:10.726981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fe582454-6947-482a-b911-aa1c06534da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:10.764058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:10.838668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 988/1944] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:10.862674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:10.866710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:10.975406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d47c2b-e13c-460a-84e4-5d289a0bd3cb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:10.989419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faaf10a-fc9b-49b0-9123-c74cf107240a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:10.990829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf10a-fc9b-49b0-9123-c74cf107240a service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:10.991122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf10a-fc9b-49b0-9123-c74cf107240a service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:10.991426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf10a-fc9b-49b0-9123-c74cf107240a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:11.005519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faaf10a-fc9b-49b0-9123-c74cf107240a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:11.009387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faaf10a-fc9b-49b0-9123-c74cf107240a service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:11.024228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 957/1945] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:11.977745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe582454-6947-482a-b911-aa1c06534da7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:12.051352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe582454-6947-482a-b911-aa1c06534da7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:12.056963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:12.057640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:12.057926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:12.058209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] PATH_INFO: `/v3/users/4305d5ed53d64205a5fa1d0588d80e13` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:12.059438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4305d5ed53d64205a5fa1d0588d80e13)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:12.063130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:12.134588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4305d5ed53d64205a5fa1d0588d80e13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:12.135819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4305d5ed53d64205a5fa1d0588d80e13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:12.148115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4305d5ed53d64205a5fa1d0588d80e13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:12.149016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4305d5ed53d64205a5fa1d0588d80e13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:12.163387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4305d5ed53d64205a5fa1d0588d80e13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:12.189579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 989/1946] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:11 2022] DELETE /identity/v3/users/4305d5ed53d64205a5fa1d0588d80e13 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:12.194373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:12.194855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:12.195130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:12.229250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:12.229250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:12.315440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 958/1947] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:12.329191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:12.333163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:12.387551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:12.413944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0975750-7366-4ed2-85a9-aa33b9cf89d7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:12.418621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:12.420049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:12.420445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:12.420823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:12.434239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:12.438721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:12.454290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 990/1948] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:12.467472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:12.469694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:12.567467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580e9bc-ecb8-4204-b99d-41d5b8b99cef None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:12.574043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:12.574759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:12.575052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:12.575361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:12.581817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:12.655799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:12.672306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:12.674628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:12.694856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 959/1949] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:12.848669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:12.851549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:12.877422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1fcd0-c710-4b32-a623-6ef55fd61f6d service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:12.884373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:12.885065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:12.885409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:12.886248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] PATH_INFO: `/v3/projects/4d228d0afa7c412fb1c0088e008edac3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:12.888134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d228d0afa7c412fb1c0088e008edac3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:12.890842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:12.987256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d228d0afa7c412fb1c0088e008edac3', 'request_id': 'req-fd6d4094-e2ea-417d-b689-cb1d36abd497'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.001407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d228d0afa7c412fb1c0088e008edac3', 'request_id': 'req-fd6d4094-e2ea-417d-b689-cb1d36abd497'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.010749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d228d0afa7c412fb1c0088e008edac3', 'request_id': 'req-fd6d4094-e2ea-417d-b689-cb1d36abd497'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.036712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] The token cache is being invalidate because project 4d228d0afa7c412fb1c0088e008edac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:13.037022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.039154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.048647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 991/1950] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:12 2022] DELETE /identity/v3/projects/4d228d0afa7c412fb1c0088e008edac3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:13.049916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:13.076270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91b82f-8bf5-48d7-90b1-bba3545ca7ee service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:13.165427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:13.166130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:13.166647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:13.166775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] PATH_INFO: `/v3/users/7d268785bf4d441d95724e249f7b06a9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:13.167884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d268785bf4d441d95724e249f7b06a9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:13.170001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:13.257126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d268785bf4d441d95724e249f7b06a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.271707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d268785bf4d441d95724e249f7b06a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.287424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d268785bf4d441d95724e249f7b06a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.288832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d268785bf4d441d95724e249f7b06a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.289520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f8e6d76e-d73a-4bb3-a163-1e17d59de9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d268785bf4d441d95724e249f7b06a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:13.312993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 960/1951] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:13 2022] DELETE /identity/v3/users/7d268785bf4d441d95724e249f7b06a9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:13.842659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:13.844918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:13.872079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d4094-e2ea-417d-b689-cb1d36abd497 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:13.939255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:13.939941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:13.940225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:13.940680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] PATH_INFO: `/v3/projects/5a2a36368bad459f869fb2898aa33510` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:13.941714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a2a36368bad459f869fb2898aa33510)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:13.944108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:13.997919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2a36368bad459f869fb2898aa33510', 'request_id': 'req-e125fb9a-7cfe-4410-9459-b10c036a96a1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:14.004789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2a36368bad459f869fb2898aa33510', 'request_id': 'req-e125fb9a-7cfe-4410-9459-b10c036a96a1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:14.010388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:14.012636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2a36368bad459f869fb2898aa33510', 'request_id': 'req-e125fb9a-7cfe-4410-9459-b10c036a96a1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:14.029776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] The token cache is being invalidate because project 5a2a36368bad459f869fb2898aa33510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:14.030224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:14.031051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e125fb9a-7cfe-4410-9459-b10c036a96a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:14.032690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 992/1952] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:13 2022] DELETE /identity/v3/projects/5a2a36368bad459f869fb2898aa33510 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:14.037902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d4397-5bc6-40b8-880f-a8244bb90244 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.038197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d4397-5bc6-40b8-880f-a8244bb90244 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.038487 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d4397-5bc6-40b8-880f-a8244bb90244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.065711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2b7d4397-5bc6-40b8-880f-a8244bb90244 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:14.066981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2b7d4397-5bc6-40b8-880f-a8244bb90244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:14.141871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 961/1953] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:14.151115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57087b-5017-4549-b954-f1439b8e51cd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.151730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57087b-5017-4549-b954-f1439b8e51cd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.152014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57087b-5017-4549-b954-f1439b8e51cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.172704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fb57087b-5017-4549-b954-f1439b8e51cd None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:14.174143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fb57087b-5017-4549-b954-f1439b8e51cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:14.187062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 993/1954] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:14.188480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d4397-5bc6-40b8-880f-a8244bb90244 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.200216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d4397-5bc6-40b8-880f-a8244bb90244 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.205164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.205895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.206356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.206556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.207236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:14.207545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.209209 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.220845 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:14.227731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 962/1955] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:44:14.231752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57087b-5017-4549-b954-f1439b8e51cd None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.242929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57087b-5017-4549-b954-f1439b8e51cd None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.250409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.251055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.251535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.251973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.252514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.254677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.286354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 994/1956] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:14 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:14.291392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.304181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623e480-2393-43fc-891c-158d704fa986 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.381257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.381952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.382250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.383801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.387830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.390005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.425348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 963/1957] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:14 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:14.429700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.441599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e668a7-e3a6-4bbd-86a9-63fe76ae60d6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.447598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.448321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.448680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.448960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.449632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.451291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.463619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 995/1958] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:14.468248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.480812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698146d-da05-479d-b223-945cd4fde1f2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.485888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679d760-2344-4b43-b9a8-ad750aae0486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.486623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679d760-2344-4b43-b9a8-ad750aae0486 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.486967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679d760-2344-4b43-b9a8-ad750aae0486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.487759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679d760-2344-4b43-b9a8-ad750aae0486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.488125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679d760-2344-4b43-b9a8-ad750aae0486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.489807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679d760-2344-4b43-b9a8-ad750aae0486 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.502784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 964/1959] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:14.507596 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.519304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169205d-1ded-4f78-999b-c32d9f32ee11 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.524054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.525646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.525646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.525646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] PATH_INFO: `/v3/projects/2dd3aa08031747c083e6734e45584f9b/users/06bf8676b1f04ef6bd15a38a76763408/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.543271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd3aa08031747c083e6734e45584f9b, user_id=06bf8676b1f04ef6bd15a38a76763408, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.546015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.564339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 996/1960] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:14 2022] PUT /identity/v3/projects/2dd3aa08031747c083e6734e45584f9b/users/06bf8676b1f04ef6bd15a38a76763408/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:14.577063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441faf3a-ffc3-45cc-a69c-5e40e5f58dc4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.577359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441faf3a-ffc3-45cc-a69c-5e40e5f58dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.577629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441faf3a-ffc3-45cc-a69c-5e40e5f58dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.614010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-441faf3a-ffc3-45cc-a69c-5e40e5f58dc4 None None] MFA Rules not processed for user `06bf8676b1f04ef6bd15a38a76763408`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:14.615229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-441faf3a-ffc3-45cc-a69c-5e40e5f58dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:14.692141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 965/1961] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:14.697805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.768517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f84d-13b9-463a-90ad-937200d6a87a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.775459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.776083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.776365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.776641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.777294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.779475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.816744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 997/1962] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:14 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:14.818030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441faf3a-ffc3-45cc-a69c-5e40e5f58dc4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.832496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441faf3a-ffc3-45cc-a69c-5e40e5f58dc4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.907813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.908471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.908756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.909035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.909929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.912049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.923376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:14.947829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 966/1963] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:14 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:14.951668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.963145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a0e19-408e-40d8-8543-0a13d8f9e70b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.972727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.973387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.973684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.974016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.974630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.976292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.995589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 998/1964] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:14.997304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:15.009759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f397f7-f22f-4873-a640-1bf7a3a2a197 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:15.016155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:15.016810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.017089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.017373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] PATH_INFO: `/v3/projects/37fa2c42449e4d11afb70cbb3b178661/users/7ca0d6392a144263b350a57b00de1cf2/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.033231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fa2c42449e4d11afb70cbb3b178661, user_id=7ca0d6392a144263b350a57b00de1cf2, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:15.036260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:15.052252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 967/1965] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:14 2022] PUT /identity/v3/projects/37fa2c42449e4d11afb70cbb3b178661/users/7ca0d6392a144263b350a57b00de1cf2/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:15.056411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:15.075180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d0a28-3c5b-48c5-8c92-8b2836f855a4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:15.134890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f516-aac0-4a03-b881-189ec3923f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:15.135546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f516-aac0-4a03-b881-189ec3923f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.135827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f516-aac0-4a03-b881-189ec3923f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.136106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f516-aac0-4a03-b881-189ec3923f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.136761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f516-aac0-4a03-b881-189ec3923f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:15.138845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f516-aac0-4a03-b881-189ec3923f68 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:15.139943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:15.155685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 999/1966] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:15.157544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:15.169902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d22ca-7823-45f9-8b17-595e281e6adf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:15.174698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:15.175403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.175692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.175971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] PATH_INFO: `/v3/projects/37fa2c42449e4d11afb70cbb3b178661/users/7ca0d6392a144263b350a57b00de1cf2/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.177921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fa2c42449e4d11afb70cbb3b178661, user_id=7ca0d6392a144263b350a57b00de1cf2, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:15.180832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:15.192906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 968/1967] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:15 2022] PUT /identity/v3/projects/37fa2c42449e4d11afb70cbb3b178661/users/7ca0d6392a144263b350a57b00de1cf2/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:15.204347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.204639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.204918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.237522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] MFA Rules not processed for user `7ca0d6392a144263b350a57b00de1cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:15.239099 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:15.293969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:15.296130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:15.314496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1000/1968] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:15.326488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:15.328004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:15.363770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45838dc0-37fc-4671-8929-677dd43a3270 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:15.368908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:15.369560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.369975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.370256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] PATH_INFO: `/v3/users/38a6a563ffb64118be7a7f8ea0eec755` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.371443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a6a563ffb64118be7a7f8ea0eec755)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:15.373382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:15.386196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c602-0925-4369-97af-c514828de5fd None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:15.390176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb0b21-6292-4973-b417-c628bf7b4fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:15.390784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0b21-6292-4973-b417-c628bf7b4fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.391062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0b21-6292-4973-b417-c628bf7b4fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.391370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb0b21-6292-4973-b417-c628bf7b4fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.402004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb0b21-6292-4973-b417-c628bf7b4fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:15.403497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb0b21-6292-4973-b417-c628bf7b4fd6 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:15.413748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1001/1969] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:15.424098 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a6a563ffb64118be7a7f8ea0eec755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:15.432854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a6a563ffb64118be7a7f8ea0eec755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:15.444483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a6a563ffb64118be7a7f8ea0eec755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:15.445591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a6a563ffb64118be7a7f8ea0eec755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:15.447948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a6a563ffb64118be7a7f8ea0eec755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:15.489651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 969/1970] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:15 2022] DELETE /identity/v3/users/38a6a563ffb64118be7a7f8ea0eec755 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:15.494138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.494536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.494883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.522274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:15.523647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:15.526228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:15.527694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:15.573619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:15.595316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1002/1971] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:15.605539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee5b5a-120f-4472-ab52-c0fa2b619fb0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:15.606102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:15.608477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:15.612177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:15.613322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.613892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.614460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.617248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:15.655570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:15.698025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982874c8-5112-4e60-a515-d194a94660f3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:15.701888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:15.702546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:15.702808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:15.703222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:15.716929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:15.719452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:15.719781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:15.722749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:15.734570 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 970/1972] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:15.735676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1003/1973] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:16.102548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.104535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.119270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3755-3dd5-4e15-addc-715464487e3f service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:16.124134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:16.124894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:16.125249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:16.125604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] PATH_INFO: `/v3/users/06bf8676b1f04ef6bd15a38a76763408` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:16.126751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] RBAC: Authorizing `identity:delete_user(user_id=06bf8676b1f04ef6bd15a38a76763408)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:16.128742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:16.182307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06bf8676b1f04ef6bd15a38a76763408'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.189299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.191326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.194667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06bf8676b1f04ef6bd15a38a76763408'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.204749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06bf8676b1f04ef6bd15a38a76763408'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.205503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d01bed-8eab-4e55-a99b-d81caed93b3c service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:16.205639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06bf8676b1f04ef6bd15a38a76763408'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.206683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e81c429e-1299-4d94-9242-92bafd6ad07f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06bf8676b1f04ef6bd15a38a76763408'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.220048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:16.220444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:16.220723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:16.220998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] PATH_INFO: `/v3/projects/9888d2b17cfa4a53940c89e0a4964127` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:16.222157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9888d2b17cfa4a53940c89e0a4964127)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:16.224123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:16.249015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 971/1974] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:16 2022] DELETE /identity/v3/users/06bf8676b1f04ef6bd15a38a76763408 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:16.253509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:16.253803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:16.254084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:16.273916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:16.275824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.279210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9888d2b17cfa4a53940c89e0a4964127', 'request_id': 'req-d984f86a-95ca-47e2-949a-3fa53657b9f6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.286003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9888d2b17cfa4a53940c89e0a4964127', 'request_id': 'req-d984f86a-95ca-47e2-949a-3fa53657b9f6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.291614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9888d2b17cfa4a53940c89e0a4964127', 'request_id': 'req-d984f86a-95ca-47e2-949a-3fa53657b9f6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.304810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] The token cache is being invalidate because project 9888d2b17cfa4a53940c89e0a4964127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:16.305069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.305780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.307576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1004/1975] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:16 2022] DELETE /identity/v3/projects/9888d2b17cfa4a53940c89e0a4964127 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:16.311295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.313072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.346768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 972/1976] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:16.356836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.358261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.363477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:16.378941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f86a-95ca-47e2-949a-3fa53657b9f6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:16.383911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:16.384582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:16.384872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:16.385155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] PATH_INFO: `/v3/users/c20676affe6f457a865c8a17442a6e55` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:16.386310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20676affe6f457a865c8a17442a6e55)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:16.388355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:16.419183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda46de-4c63-4415-9766-52f020a38b87 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:16.423041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:16.423692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:16.423973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:16.424242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:16.433132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:16.434638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:16.435898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20676affe6f457a865c8a17442a6e55'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.437079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20676affe6f457a865c8a17442a6e55'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.441754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:16.444453 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20676affe6f457a865c8a17442a6e55'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.445679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20676affe6f457a865c8a17442a6e55'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.445810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 973/1977] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:16.454455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20676affe6f457a865c8a17442a6e55'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.473770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1005/1978] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:16 2022] DELETE /identity/v3/users/c20676affe6f457a865c8a17442a6e55 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:16.729672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.731347 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.749522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d153-bbf3-42d0-b130-a4890d64650d service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:16.784853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.787089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.828788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:16.829467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:16.829745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:16.830781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] PATH_INFO: `/v3/projects/2dd3aa08031747c083e6734e45584f9b` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:16.831144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd3aa08031747c083e6734e45584f9b)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:16.833245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:16.860191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dae22-37fb-4ac3-a955-ab2676bbf384 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:16.865655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:16.866302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:16.866589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:16.866863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] PATH_INFO: `/v3/projects/2dd31875cb194c91a2f734f7d4e29108` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:16.868082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd31875cb194c91a2f734f7d4e29108)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:16.870040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:16.885465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd3aa08031747c083e6734e45584f9b', 'request_id': 'req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.891079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd3aa08031747c083e6734e45584f9b', 'request_id': 'req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.898214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd3aa08031747c083e6734e45584f9b', 'request_id': 'req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.910795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] The token cache is being invalidate because project 2dd3aa08031747c083e6734e45584f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:16.911096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.911755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.913408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 974/1979] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:16 2022] DELETE /identity/v3/projects/2dd3aa08031747c083e6734e45584f9b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:16.917469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.918813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.920524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd31875cb194c91a2f734f7d4e29108', 'request_id': 'req-34934c60-86dc-4511-bd81-6207c95712d5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.927001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd31875cb194c91a2f734f7d4e29108', 'request_id': 'req-34934c60-86dc-4511-bd81-6207c95712d5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.933293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd31875cb194c91a2f734f7d4e29108', 'request_id': 'req-34934c60-86dc-4511-bd81-6207c95712d5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.949781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] The token cache is being invalidate because project 2dd31875cb194c91a2f734f7d4e29108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:16.950116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.950938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:16.953254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1006/1980] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:16 2022] DELETE /identity/v3/projects/2dd31875cb194c91a2f734f7d4e29108 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:16.956860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:16.958188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:16.959447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:16.990668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f294c8f-e484-4d2d-8523-c6e3e8fa233a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.031256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34934c60-86dc-4511-bd81-6207c95712d5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.037114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.037430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.037718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.038140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] PATH_INFO: `/v3/users/18dd6751b0fa4d18a06f9d5bbe0f0746` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.039359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18dd6751b0fa4d18a06f9d5bbe0f0746)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.042042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.043857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:17.069486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.070256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.070553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.070880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] PATH_INFO: `/v3/users/7ca0d6392a144263b350a57b00de1cf2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.072682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca0d6392a144263b350a57b00de1cf2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.074734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.096811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18dd6751b0fa4d18a06f9d5bbe0f0746'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.097605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18dd6751b0fa4d18a06f9d5bbe0f0746'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.111907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18dd6751b0fa4d18a06f9d5bbe0f0746'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.112886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18dd6751b0fa4d18a06f9d5bbe0f0746'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.121806 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18dd6751b0fa4d18a06f9d5bbe0f0746'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.130255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca0d6392a144263b350a57b00de1cf2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.138639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca0d6392a144263b350a57b00de1cf2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.143376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1007/1981] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:16 2022] DELETE /identity/v3/users/18dd6751b0fa4d18a06f9d5bbe0f0746 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:17.150960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca0d6392a144263b350a57b00de1cf2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.151701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca0d6392a144263b350a57b00de1cf2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.152388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca0d6392a144263b350a57b00de1cf2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.171720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 975/1982] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:16 2022] DELETE /identity/v3/users/7ca0d6392a144263b350a57b00de1cf2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:17.545885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:17.547568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:17.589791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:17.591925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:17.629686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df9066-3688-4065-9a14-6a37fec78074 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.667050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8decc-acce-4352-9ee9-47adcb34bd61 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.672211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.672925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.673276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.673556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] PATH_INFO: `/v3/projects/37fa2c42449e4d11afb70cbb3b178661` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.674667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37fa2c42449e4d11afb70cbb3b178661)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.676619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.679183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:17.691035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.691797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.692087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.692369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] PATH_INFO: `/v3/projects/2c0c28a3c5704898b43a1dd7ef68a5a2` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.693567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0c28a3c5704898b43a1dd7ef68a5a2)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.695639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.731362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:17.734435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37fa2c42449e4d11afb70cbb3b178661', 'request_id': 'req-35ca939e-39e2-4b4b-96cc-3a431012d1f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.740218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37fa2c42449e4d11afb70cbb3b178661', 'request_id': 'req-35ca939e-39e2-4b4b-96cc-3a431012d1f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.746799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37fa2c42449e4d11afb70cbb3b178661', 'request_id': 'req-35ca939e-39e2-4b4b-96cc-3a431012d1f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.749927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0c28a3c5704898b43a1dd7ef68a5a2', 'request_id': 'req-779b14b5-0bf0-4984-b06c-719e7b54f260'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.757554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0c28a3c5704898b43a1dd7ef68a5a2', 'request_id': 'req-779b14b5-0bf0-4984-b06c-719e7b54f260'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.760844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] The token cache is being invalidate because project 37fa2c42449e4d11afb70cbb3b178661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:17.761144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.761924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-35ca939e-39e2-4b4b-96cc-3a431012d1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.763523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 976/1983] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:17 2022] DELETE /identity/v3/projects/37fa2c42449e4d11afb70cbb3b178661 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:17.763711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0c28a3c5704898b43a1dd7ef68a5a2', 'request_id': 'req-779b14b5-0bf0-4984-b06c-719e7b54f260'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.768370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73eea9-86d1-4e44-90c2-3836bfa20499 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.768713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73eea9-86d1-4e44-90c2-3836bfa20499 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.768994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73eea9-86d1-4e44-90c2-3836bfa20499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.777417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] The token cache is being invalidate because project 2c0c28a3c5704898b43a1dd7ef68a5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:17.777710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.779139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-779b14b5-0bf0-4984-b06c-719e7b54f260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:17.780684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1008/1984] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:17 2022] DELETE /identity/v3/projects/2c0c28a3c5704898b43a1dd7ef68a5a2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:17.786348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d74e56-763e-478e-8118-e1c4b2af7c17 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.786636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d74e56-763e-478e-8118-e1c4b2af7c17 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.786912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d74e56-763e-478e-8118-e1c4b2af7c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.789464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fe73eea9-86d1-4e44-90c2-3836bfa20499 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:17.791002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fe73eea9-86d1-4e44-90c2-3836bfa20499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:17.806842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-25d74e56-763e-478e-8118-e1c4b2af7c17 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:17.808053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-25d74e56-763e-478e-8118-e1c4b2af7c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:17.856692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 977/1985] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:17.864260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436675e8-1261-4abd-88f2-1520eefa6224 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.864551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436675e8-1261-4abd-88f2-1520eefa6224 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.864829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436675e8-1261-4abd-88f2-1520eefa6224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.871406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1009/1986] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:17.879641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fca4f1-72cc-4254-b2ba-80e16cee200c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.879992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fca4f1-72cc-4254-b2ba-80e16cee200c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.880242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fca4f1-72cc-4254-b2ba-80e16cee200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.884839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-436675e8-1261-4abd-88f2-1520eefa6224 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:17.886042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-436675e8-1261-4abd-88f2-1520eefa6224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:17.893994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 978/1987] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:17.898138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436675e8-1261-4abd-88f2-1520eefa6224 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:17.901984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b8fca4f1-72cc-4254-b2ba-80e16cee200c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:17.903279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b8fca4f1-72cc-4254-b2ba-80e16cee200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:17.907208 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436675e8-1261-4abd-88f2-1520eefa6224 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.911729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.915182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1010/1988] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:17.915415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.915415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.915415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.915415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:17.915415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.915917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.916029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fca4f1-72cc-4254-b2ba-80e16cee200c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:17.925054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fca4f1-72cc-4254-b2ba-80e16cee200c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.927258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 979/1989] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:44:17.930234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.930960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.931373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.931714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.932287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:17.932388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:17.932682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.934341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.942024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cbee99-03fa-482f-b7e7-a5332f7f9888 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.946729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1011/1990] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:17.946855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.947478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.947758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.948030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.948649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.950562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.951638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:17.962709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e295af-3896-48cd-b149-6b3d5b8537f1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:17.968329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:17.969005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:17.969318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:17.969634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:17.970387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:17.972731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:17.980362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 980/1991] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:17 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:17.984560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.010629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1012/1992] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:17 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:18.011605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.038750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:18.052644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de15e-c1cb-4c8b-a080-bac11fb477e3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.077815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea93fe9-a345-4398-8ef4-d761e1473472 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.082599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.083251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.083564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.083849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.084489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.086445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.114331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1013/1993] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:18 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:18.115664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.121100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.121946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.123134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.123134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.123682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.124004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:18.126139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.128244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab90cc-6158-4db5-813a-abc57fc717c5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.135727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.136544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.136888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.137239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.138033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.139912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.156185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1014/1994] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:18.160378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 981/1995] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:17 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:18.160570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.164056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.174064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37e49e-a378-41d5-af8d-83001e4c53ab admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.177720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4395d-4b6e-46e6-a3ca-222c9b0150c9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.179328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.179961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.180397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.180839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.181877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.184543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.185150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.186019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.187082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.187586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.188947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.191059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.200843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1015/1996] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:18.205019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.214287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 982/1997] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:18.221042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a05b0-407c-412e-a85b-c22fd5a57c06 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.222021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.232447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6be796-e8cb-4665-9549-0dc4d1f4678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.233281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6be796-e8cb-4665-9549-0dc4d1f4678c admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.233659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6be796-e8cb-4665-9549-0dc4d1f4678c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.234030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6be796-e8cb-4665-9549-0dc4d1f4678c admin admin] PATH_INFO: `/v3/projects/9cea29bac8a543af83270d0e44ae4624/users/2ee4a7c49bd34e61a36f22bacb035e44/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.236458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc5a17-f7bc-4cd7-9637-f7239378a0a4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.241581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.242464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.243051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.243620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.244622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.246256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.253756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6be796-e8cb-4665-9549-0dc4d1f4678c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cea29bac8a543af83270d0e44ae4624, user_id=2ee4a7c49bd34e61a36f22bacb035e44, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.256970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6be796-e8cb-4665-9549-0dc4d1f4678c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.258498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 983/1998] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:18.263614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.274940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fde6d7-42af-4018-a82b-7d0cfaac71c0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.275742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1016/1999] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:18 2022] PUT /identity/v3/projects/9cea29bac8a543af83270d0e44ae4624/users/2ee4a7c49bd34e61a36f22bacb035e44/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:18.287699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299406a-31fc-4944-b5d7-72a7686aed51 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.288067 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299406a-31fc-4944-b5d7-72a7686aed51 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.288420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299406a-31fc-4944-b5d7-72a7686aed51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.320290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e299406a-31fc-4944-b5d7-72a7686aed51 None None] MFA Rules not processed for user `2ee4a7c49bd34e61a36f22bacb035e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:18.321944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e299406a-31fc-4944-b5d7-72a7686aed51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:18.339369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f2e02-bba3-4a11-a70b-c686be26fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.340299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f2e02-bba3-4a11-a70b-c686be26fec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.340588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f2e02-bba3-4a11-a70b-c686be26fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.340865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f2e02-bba3-4a11-a70b-c686be26fec4 admin admin] PATH_INFO: `/v3/projects/b1d68e8e7e104bccb721e3c036a7907a/users/686bbf803837400e94f1df1efd04a407/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.356528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f2e02-bba3-4a11-a70b-c686be26fec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d68e8e7e104bccb721e3c036a7907a, user_id=686bbf803837400e94f1df1efd04a407, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.359450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f2e02-bba3-4a11-a70b-c686be26fec4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.373387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 984/2000] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:18 2022] PUT /identity/v3/projects/b1d68e8e7e104bccb721e3c036a7907a/users/686bbf803837400e94f1df1efd04a407/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:18.378137 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.378463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.378769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.390252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1017/2001] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:18.395035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299406a-31fc-4944-b5d7-72a7686aed51 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.400348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:18.401587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:18.406560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299406a-31fc-4944-b5d7-72a7686aed51 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.447835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:18.472873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 985/2002] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:18.473104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.473104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.473104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.473104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.473438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.475369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.480047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.481438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:18.511158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1018/2003] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:18 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:18.512774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.548315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88906d8f-7012-424f-9b1e-bf81ee78f9cb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.549944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:18.586442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cacad-521d-44f1-8ed3-78fbcffbfaa9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.592072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.592710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.592999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.593432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.594016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.596125 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.604227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.605096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.605380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.605659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.614988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.616560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.626753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 986/2004] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:18.632809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1019/2005] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:18 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:18.634316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.644592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4c750-78d8-41af-a571-1ef66cd59d86 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.650290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.650948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.651290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.651574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.652294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.654038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.664470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 987/2006] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:18.668819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.678997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efe1b9-2f67-41a3-a7e9-bd1659b74ba0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.683965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.684654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.684946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.685227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] PATH_INFO: `/v3/projects/4bf7bf496a444324a50782bd2573e53d/users/edf31778c4934bd494acb0d8d5c3c710/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.701544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf7bf496a444324a50782bd2573e53d, user_id=edf31778c4934bd494acb0d8d5c3c710, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.704435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.719306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1020/2007] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:18 2022] PUT /identity/v3/projects/4bf7bf496a444324a50782bd2573e53d/users/edf31778c4934bd494acb0d8d5c3c710/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:18.723665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.735736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee2ca2-5b5a-4ac8-a181-40b96d8b2e8c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.801754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8545661-8271-4da3-ba65-aac7cb97fe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.802513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545661-8271-4da3-ba65-aac7cb97fe1c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.802677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545661-8271-4da3-ba65-aac7cb97fe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.802929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8545661-8271-4da3-ba65-aac7cb97fe1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.803645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8545661-8271-4da3-ba65-aac7cb97fe1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.805272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8545661-8271-4da3-ba65-aac7cb97fe1c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.816878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 988/2008] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:18.821358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:18.832000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f55f12-001f-4a08-a14c-218fa0d28740 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:18.837290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:18.838149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.838439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.838720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] PATH_INFO: `/v3/projects/4bf7bf496a444324a50782bd2573e53d/users/edf31778c4934bd494acb0d8d5c3c710/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.840822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf7bf496a444324a50782bd2573e53d, user_id=edf31778c4934bd494acb0d8d5c3c710, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:18.843562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:18.854749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1021/2009] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:18 2022] PUT /identity/v3/projects/4bf7bf496a444324a50782bd2573e53d/users/edf31778c4934bd494acb0d8d5c3c710/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:18.867401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125ce54-6c79-4be3-b23f-a0ca55763a92 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:18.867776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125ce54-6c79-4be3-b23f-a0ca55763a92 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:18.868256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125ce54-6c79-4be3-b23f-a0ca55763a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:18.904068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3125ce54-6c79-4be3-b23f-a0ca55763a92 None None] MFA Rules not processed for user `edf31778c4934bd494acb0d8d5c3c710`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:18.905284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3125ce54-6c79-4be3-b23f-a0ca55763a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:18.928163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:18.997160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 989/2010] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:19.008412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:19.010993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:19.076187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70af33-9d0f-48e7-bc6f-1d112134430c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:19.079774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776fa9e-24a3-47d4-9bfe-a83630e2dd24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:19.080391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fa9e-24a3-47d4-9bfe-a83630e2dd24 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:19.080669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fa9e-24a3-47d4-9bfe-a83630e2dd24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:19.080939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776fa9e-24a3-47d4-9bfe-a83630e2dd24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:19.090694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776fa9e-24a3-47d4-9bfe-a83630e2dd24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:19.092185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776fa9e-24a3-47d4-9bfe-a83630e2dd24 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:19.101687 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1022/2011] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:19.245983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125ce54-6c79-4be3-b23f-a0ca55763a92 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:19.280694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da24790-249c-455e-8b44-2b734d5132a5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:19.280694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da24790-249c-455e-8b44-2b734d5132a5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:19.280694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da24790-249c-455e-8b44-2b734d5132a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:19.330209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0da24790-249c-455e-8b44-2b734d5132a5 None None] MFA Rules not processed for user `686bbf803837400e94f1df1efd04a407`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:19.331519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0da24790-249c-455e-8b44-2b734d5132a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:19.344685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125ce54-6c79-4be3-b23f-a0ca55763a92 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:19.349916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:19.350681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:19.351068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:19.351609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] PATH_INFO: `/v3/users/2ee4a7c49bd34e61a36f22bacb035e44` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:19.353397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee4a7c49bd34e61a36f22bacb035e44)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:19.355628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:19.420638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee4a7c49bd34e61a36f22bacb035e44'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:19.423256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1023/2012] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:19.434420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee4a7c49bd34e61a36f22bacb035e44'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:19.436155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da24790-249c-455e-8b44-2b734d5132a5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:19.448317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da24790-249c-455e-8b44-2b734d5132a5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:19.448676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee4a7c49bd34e61a36f22bacb035e44'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:19.449641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee4a7c49bd34e61a36f22bacb035e44'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:19.450391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cb03fc77-938f-4c54-a0ea-c4282da88078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee4a7c49bd34e61a36f22bacb035e44'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:19.452896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:19.453700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:19.454131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:19.454568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:19.456551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:19.468845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:19.470788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:19.472235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 990/2013] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:19 2022] DELETE /identity/v3/users/2ee4a7c49bd34e61a36f22bacb035e44 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:19.476820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:19.477470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:19.477693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:19.485401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1024/2014] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:19.497374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:19.498826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:19.533241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:19.568303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 991/2015] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:19.575119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:19.576642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:19.583298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:19.644589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532acbf-62f1-41cc-b8a0-c626e087e808 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:19.741005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:19.741785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:19.742285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:19.742583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:19.756674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:19.758479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:19.779544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1025/2016] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:20.086336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:20.088913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:20.102268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcbe9e-b30c-4934-8284-f3b5b50884d6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:20.114980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:20.115711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:20.115983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:20.116264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] PATH_INFO: `/v3/projects/9cea29bac8a543af83270d0e44ae4624` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:20.117442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cea29bac8a543af83270d0e44ae4624)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:20.119463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:20.171136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cea29bac8a543af83270d0e44ae4624', 'request_id': 'req-e153b975-c745-4a49-9d1d-ab354067ebd5'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.177143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cea29bac8a543af83270d0e44ae4624', 'request_id': 'req-e153b975-c745-4a49-9d1d-ab354067ebd5'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.184240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cea29bac8a543af83270d0e44ae4624', 'request_id': 'req-e153b975-c745-4a49-9d1d-ab354067ebd5'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.199616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] The token cache is being invalidate because project 9cea29bac8a543af83270d0e44ae4624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:20.200757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.200757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.202236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 992/2017] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:20 2022] DELETE /identity/v3/projects/9cea29bac8a543af83270d0e44ae4624 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:20.206349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:20.216461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de2eed-2dd4-4cbf-a9d7-6a75290a5d16 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:20.285107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:20.285876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:20.286162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:20.286449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] PATH_INFO: `/v3/users/edf31778c4934bd494acb0d8d5c3c710` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:20.287725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf31778c4934bd494acb0d8d5c3c710)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:20.290465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:20.373502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf31778c4934bd494acb0d8d5c3c710'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.374251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf31778c4934bd494acb0d8d5c3c710'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.391174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf31778c4934bd494acb0d8d5c3c710'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.391963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf31778c4934bd494acb0d8d5c3c710'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.401734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-68b936e7-9dfa-4bcd-aa67-6f28e8f44b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf31778c4934bd494acb0d8d5c3c710'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.404567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:20.431238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1026/2018] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:20 2022] DELETE /identity/v3/users/edf31778c4934bd494acb0d8d5c3c710 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:20.733065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:20.734578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:20.749320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153b975-c745-4a49-9d1d-ab354067ebd5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:20.816231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:20.816881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:20.817161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:20.817429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] PATH_INFO: `/v3/projects/4bf7bf496a444324a50782bd2573e53d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:20.818519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf7bf496a444324a50782bd2573e53d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:20.820700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:20.879378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf7bf496a444324a50782bd2573e53d', 'request_id': 'req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.889363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf7bf496a444324a50782bd2573e53d', 'request_id': 'req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.897193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf7bf496a444324a50782bd2573e53d', 'request_id': 'req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.901835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:20.928578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] The token cache is being invalidate because project 4bf7bf496a444324a50782bd2573e53d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:20.928848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.930151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6ffe970d-97d4-4d8e-8afe-7cddcf76b8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:20.935816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 993/2019] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:20 2022] DELETE /identity/v3/projects/4bf7bf496a444324a50782bd2573e53d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:20.938438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a763b1-5fa9-4510-8146-c2c7a284e4e4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:20.939064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a763b1-5fa9-4510-8146-c2c7a284e4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:20.939064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a763b1-5fa9-4510-8146-c2c7a284e4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:20.962975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e5a763b1-5fa9-4510-8146-c2c7a284e4e4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:20.965025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e5a763b1-5fa9-4510-8146-c2c7a284e4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:20.970591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f93e-e607-4daf-bdbd-e0720de123cf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:20.970967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f93e-e607-4daf-bdbd-e0720de123cf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:20.971402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f93e-e607-4daf-bdbd-e0720de123cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:20.992440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a568f93e-e607-4daf-bdbd-e0720de123cf None None] MFA Rules not processed for user `686bbf803837400e94f1df1efd04a407`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:20.994695 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a568f93e-e607-4daf-bdbd-e0720de123cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:21.039128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1027/2020] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:21.046823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d185b7a-543c-44c4-aaae-f313de7aa4ac None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.047211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d185b7a-543c-44c4-aaae-f313de7aa4ac None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.047572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d185b7a-543c-44c4-aaae-f313de7aa4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.069027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2d185b7a-543c-44c4-aaae-f313de7aa4ac None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:21.070244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2d185b7a-543c-44c4-aaae-f313de7aa4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:21.073488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 994/2021] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:21.081556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1028/2022] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:21.082643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f93e-e607-4daf-bdbd-e0720de123cf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.093349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f93e-e607-4daf-bdbd-e0720de123cf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.098047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.098596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.098879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.099156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.099849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:21.100157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.101754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:21.117502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 995/2023] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:21.118997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d185b7a-543c-44c4-aaae-f313de7aa4ac None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.128518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d185b7a-543c-44c4-aaae-f313de7aa4ac None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.133780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.134427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.134710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.134990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.135704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.137786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:21.167952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1029/2024] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:21 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:21.172443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.183130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188e5a0-e320-4774-a734-dc6377afa267 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.247818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.248439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.248718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.248986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.249620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.251579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:21.275451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 996/2025] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:21 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:21.279577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.290579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45202-b975-4cf8-80f0-99e6c87e7908 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.295552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.296262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.296600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.296969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.297700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.299413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:21.310786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1030/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:21.315376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.325533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e89ae-14f3-4faa-95cc-af6b8b6eb797 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.330166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b007a-5254-4788-93af-23a56603523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.330781 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b007a-5254-4788-93af-23a56603523a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.331067 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b007a-5254-4788-93af-23a56603523a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.331383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b007a-5254-4788-93af-23a56603523a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.332072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b007a-5254-4788-93af-23a56603523a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.333736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b007a-5254-4788-93af-23a56603523a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:21.351277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 997/2027] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:21.352804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.363493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd459c-9d8c-47a2-b33a-1bad3a4c8d39 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.369295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.370092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.370382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.370663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] PATH_INFO: `/v3/projects/1cd9c1a2742946ddbc49ea3ee65dd42c/users/891fc636854d4b67afdd1af156970fb4/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.388200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd9c1a2742946ddbc49ea3ee65dd42c, user_id=891fc636854d4b67afdd1af156970fb4, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.391061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:21.409639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1031/2028] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:21 2022] PUT /identity/v3/projects/1cd9c1a2742946ddbc49ea3ee65dd42c/users/891fc636854d4b67afdd1af156970fb4/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:21.422468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e6d18-2364-4e2d-85b5-ec9aa44f7998 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.423278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e6d18-2364-4e2d-85b5-ec9aa44f7998 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.423794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e6d18-2364-4e2d-85b5-ec9aa44f7998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.518995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1a5e6d18-2364-4e2d-85b5-ec9aa44f7998 None None] MFA Rules not processed for user `891fc636854d4b67afdd1af156970fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:21.520826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1a5e6d18-2364-4e2d-85b5-ec9aa44f7998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:21.607629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:21.642559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 998/2029] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:21.648355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.793294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e2a3a-1d19-4785-a4f8-75a9e589d2e4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.804681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.806563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.806996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.807459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.808283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.810839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:21.858418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1032/2030] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:21 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:21.862933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e6d18-2364-4e2d-85b5-ec9aa44f7998 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:21.884100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e6d18-2364-4e2d-85b5-ec9aa44f7998 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:21.976881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:21.977832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:21.978247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:21.978732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:21.979575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:21.982099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:22.024263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 999/2031] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:21 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:22.029899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:22.039082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:22.044195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11405a6d-d6c7-4088-a81f-b96df5f49d8b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:22.050555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:22.051775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:22.052558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:22.052874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:22.054155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:22.056731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:22.079171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1033/2032] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:22.084809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:22.106426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252b7c-d1dd-484d-9072-e041fdedc5c0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:22.114993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:22.115716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:22.116000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:22.116279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] PATH_INFO: `/v3/projects/69d2e86849374070bd35695270948f71/users/706983bd13dc483da8c8099821c5ddfb/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:22.151961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d2e86849374070bd35695270948f71, user_id=706983bd13dc483da8c8099821c5ddfb, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:22.154676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:22.176974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1000/2033] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:22 2022] PUT /identity/v3/projects/69d2e86849374070bd35695270948f71/users/706983bd13dc483da8c8099821c5ddfb/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:22.182084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:22.192951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c96b-6b22-4568-8142-7b79f976fe10 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:22.264262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc56dab-5ef8-4d77-a448-9e6e4b642e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:22.264891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56dab-5ef8-4d77-a448-9e6e4b642e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:22.265183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56dab-5ef8-4d77-a448-9e6e4b642e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:22.265462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc56dab-5ef8-4d77-a448-9e6e4b642e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:22.266124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc56dab-5ef8-4d77-a448-9e6e4b642e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:22.267760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc56dab-5ef8-4d77-a448-9e6e4b642e90 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:22.279100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1034/2034] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:22.283353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:22.294604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b23bf8-8547-4b6f-a781-0f5b9c9bb203 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:22.299260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:22.299922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:22.300217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:22.300495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] PATH_INFO: `/v3/projects/69d2e86849374070bd35695270948f71/users/706983bd13dc483da8c8099821c5ddfb/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:22.302404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d2e86849374070bd35695270948f71, user_id=706983bd13dc483da8c8099821c5ddfb, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:22.305284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:22.316914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1001/2035] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:22 2022] PUT /identity/v3/projects/69d2e86849374070bd35695270948f71/users/706983bd13dc483da8c8099821c5ddfb/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:22.327740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8888d-e286-4072-92b0-3c0970e615f1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:22.328032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8888d-e286-4072-92b0-3c0970e615f1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:22.328311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8888d-e286-4072-92b0-3c0970e615f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:22.359897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e6d8888d-e286-4072-92b0-3c0970e615f1 None None] MFA Rules not processed for user `706983bd13dc483da8c8099821c5ddfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:22.361040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e6d8888d-e286-4072-92b0-3c0970e615f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:22.436128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1035/2036] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:22.449386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:22.451524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:22.514973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad12c40-2abf-4ecb-9eeb-a9bcf7fc4be2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:22.519016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:22.519632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:22.519963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:22.520251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:22.530751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:22.532230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:22.542349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1002/2037] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:22.631618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8888d-e286-4072-92b0-3c0970e615f1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:22.642362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8888d-e286-4072-92b0-3c0970e615f1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:22.646757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b98d01-5612-4545-88ed-b11ea83c416d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:22.647326 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b98d01-5612-4545-88ed-b11ea83c416d service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:22.647669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b98d01-5612-4545-88ed-b11ea83c416d service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:22.647964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b98d01-5612-4545-88ed-b11ea83c416d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:22.710493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b98d01-5612-4545-88ed-b11ea83c416d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:22.712958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b98d01-5612-4545-88ed-b11ea83c416d service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:22.722655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1036/2038] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:28.038071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:28.090035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:28.104253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597ed5c-3823-4b06-a58f-5b4141d32430 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:28.108980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:28.109615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:28.109892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:28.110163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] PATH_INFO: `/v3/users/891fc636854d4b67afdd1af156970fb4` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:28.118015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] RBAC: Authorizing `identity:delete_user(user_id=891fc636854d4b67afdd1af156970fb4)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:28.118283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:28.194375 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891fc636854d4b67afdd1af156970fb4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:28.202292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891fc636854d4b67afdd1af156970fb4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:28.223231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891fc636854d4b67afdd1af156970fb4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:28.224188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891fc636854d4b67afdd1af156970fb4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:28.224540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891fc636854d4b67afdd1af156970fb4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:28.243839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1003/2039] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:28 2022] DELETE /identity/v3/users/891fc636854d4b67afdd1af156970fb4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:28.248338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:28.248881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:28.249038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:28.268574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:28.269804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:28.279493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:28.372907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1037/2040] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:28.388615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:28.390092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:28.486263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735040b-b645-4729-9a88-f71f8d36b22a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:28.490397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:28.490987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:28.491675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:28.491675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:28.504450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:28.506081 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:28.517267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1004/2041] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:29.088694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:29.090288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:29.113553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50e2f-b962-439f-ab9f-f36e6ad45902 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:29.120048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:29.120934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:29.121283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:29.121642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] PATH_INFO: `/v3/projects/1cd9c1a2742946ddbc49ea3ee65dd42c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:29.122863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd9c1a2742946ddbc49ea3ee65dd42c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:29.125565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:29.175566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd9c1a2742946ddbc49ea3ee65dd42c', 'request_id': 'req-b3b61b0d-4758-4a62-9a08-231f230fbb16'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.181813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd9c1a2742946ddbc49ea3ee65dd42c', 'request_id': 'req-b3b61b0d-4758-4a62-9a08-231f230fbb16'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.188966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd9c1a2742946ddbc49ea3ee65dd42c', 'request_id': 'req-b3b61b0d-4758-4a62-9a08-231f230fbb16'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.203416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] The token cache is being invalidate because project 1cd9c1a2742946ddbc49ea3ee65dd42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:29.203640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.204308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.205902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1038/2042] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:29 2022] DELETE /identity/v3/projects/1cd9c1a2742946ddbc49ea3ee65dd42c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:29.215226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:29.220442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7edcf7-6211-4914-abd0-5c77fc828813 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:29.295796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:29.297352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:29.298107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:29.298806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] PATH_INFO: `/v3/users/706983bd13dc483da8c8099821c5ddfb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:29.301935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=706983bd13dc483da8c8099821c5ddfb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:29.308629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:29.324161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:29.403423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706983bd13dc483da8c8099821c5ddfb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.405973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706983bd13dc483da8c8099821c5ddfb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.415728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706983bd13dc483da8c8099821c5ddfb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.416618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706983bd13dc483da8c8099821c5ddfb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.417402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeb3261b-ce4d-4d82-99c7-c72446c35230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706983bd13dc483da8c8099821c5ddfb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:29.439826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1005/2043] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:29 2022] DELETE /identity/v3/users/706983bd13dc483da8c8099821c5ddfb => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:29.868786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:29.870372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:29.885737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b61b0d-4758-4a62-9a08-231f230fbb16 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:29.969281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:29.970130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:29.970411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:29.970686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] PATH_INFO: `/v3/projects/69d2e86849374070bd35695270948f71` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:29.971901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d2e86849374070bd35695270948f71)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:29.973855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.025467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:30.038692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d2e86849374070bd35695270948f71', 'request_id': 'req-943f5592-3984-4b29-bb61-d09f66ae0332'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:30.048987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d2e86849374070bd35695270948f71', 'request_id': 'req-943f5592-3984-4b29-bb61-d09f66ae0332'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:30.075724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d2e86849374070bd35695270948f71', 'request_id': 'req-943f5592-3984-4b29-bb61-d09f66ae0332'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:30.091686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] The token cache is being invalidate because project 69d2e86849374070bd35695270948f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:30.092128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:30.092898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-943f5592-3984-4b29-bb61-d09f66ae0332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:30.094757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1039/2044] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:29 2022] DELETE /identity/v3/projects/69d2e86849374070bd35695270948f71 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:30.099553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72829d-e956-4860-a1ed-27e4f309ac75 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.099839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72829d-e956-4860-a1ed-27e4f309ac75 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.100112 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72829d-e956-4860-a1ed-27e4f309ac75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.126327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-eb72829d-e956-4860-a1ed-27e4f309ac75 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:30.128422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eb72829d-e956-4860-a1ed-27e4f309ac75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:30.201588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1006/2045] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:30.209338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980afd7-646c-4fcc-b4a7-63c391fadc7f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.209805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980afd7-646c-4fcc-b4a7-63c391fadc7f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.210543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980afd7-646c-4fcc-b4a7-63c391fadc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.233131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7980afd7-646c-4fcc-b4a7-63c391fadc7f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:30.234607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7980afd7-646c-4fcc-b4a7-63c391fadc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:30.244543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1040/2046] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:30.249237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72829d-e956-4860-a1ed-27e4f309ac75 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:30.259993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72829d-e956-4860-a1ed-27e4f309ac75 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:30.265849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:30.266556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.266914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.267302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.268043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:30.268426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:30.270110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.286181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1007/2047] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:30.287748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980afd7-646c-4fcc-b4a7-63c391fadc7f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:30.300231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980afd7-646c-4fcc-b4a7-63c391fadc7f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:30.305766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:30.306760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.307035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.307485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.308560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:30.310489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.350370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1041/2048] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:30 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:30.351826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:30.364412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868c898-c1b6-4b0f-919d-41c70b800436 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:30.456830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:30.459076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.459602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.460075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.460871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:30.463608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.493962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1008/2049] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:30 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:30.495447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:30.506483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd06219-e63d-4101-b970-b063517bd746 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:30.511496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:30.512120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.512696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.512696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.513323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:30.514960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.526713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1042/2050] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:30.530876 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:30.537517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:30.541877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175d9c-0949-4475-914a-95f92f6eb2fe admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:30.547877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ca261-6808-491d-8a16-0bd19044914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:30.549222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca261-6808-491d-8a16-0bd19044914c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.549608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca261-6808-491d-8a16-0bd19044914c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.550089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca261-6808-491d-8a16-0bd19044914c admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.550845 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ca261-6808-491d-8a16-0bd19044914c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:30.553032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ca261-6808-491d-8a16-0bd19044914c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.566031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1009/2051] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:30.570109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:30.581156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ae57-27b4-4732-a3d4-9afe4ceb7292 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:30.585880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:30.586651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.587011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.587388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] PATH_INFO: `/v3/projects/0930b613a69749d3bc05240e88f74ccc/users/814c0227c728493ebd4364bc154cee83/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.604635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0930b613a69749d3bc05240e88f74ccc, user_id=814c0227c728493ebd4364bc154cee83, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:30.607678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.632085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09010e-53a3-48ea-ae23-7bdd7c2dbc56 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.632321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09010e-53a3-48ea-ae23-7bdd7c2dbc56 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.632645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09010e-53a3-48ea-ae23-7bdd7c2dbc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.635847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1043/2052] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:30 2022] PUT /identity/v3/projects/0930b613a69749d3bc05240e88f74ccc/users/814c0227c728493ebd4364bc154cee83/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:30.655119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4b09010e-53a3-48ea-ae23-7bdd7c2dbc56 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:30.656385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4b09010e-53a3-48ea-ae23-7bdd7c2dbc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:30.773101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1010/2053] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:30.787549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:30.791846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:30.899719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a194358-37c1-4264-866d-3dcfc8596117 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:30.905474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:30.906335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:30.906777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:30.907149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:30.929269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:30.930982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:30.947574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1044/2054] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:31.968161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:31.968718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:31.968937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:32.001741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] MFA Rules not processed for user `814c0227c728493ebd4364bc154cee83`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:32.003413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:32.098065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1011/2055] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:32.107025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:32.110433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:32.190031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337d771-3d77-4521-a70e-5020f02341ca service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:32.196070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:32.197248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:32.198574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:32.201356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:32.227073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:32.228397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:32.247401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1045/2056] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:32.336290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:32.338010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:32.369378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab786a-338c-4274-81f0-ff9295606fff None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:32.391070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:32.391774 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:32.392206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:32.392490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] PATH_INFO: `/v3/users/7f3061468e644588a9f7f8de2af94827` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:32.395631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3061468e644588a9f7f8de2af94827)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:32.398492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:32.489335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3061468e644588a9f7f8de2af94827'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:32.498714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3061468e644588a9f7f8de2af94827'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:32.511231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3061468e644588a9f7f8de2af94827'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:32.512478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3061468e644588a9f7f8de2af94827'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:32.513348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fdcc50a1-1b78-4e3a-a67f-a958a7c66337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3061468e644588a9f7f8de2af94827'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:32.539231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:32.565662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1012/2057] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:32 2022] DELETE /identity/v3/users/7f3061468e644588a9f7f8de2af94827 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:32.835988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:32.850586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11049382-359a-4f82-bcbc-64ebeaf383cd service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:32.859087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:32.927959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:32.928634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:32.928913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:32.929410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] PATH_INFO: `/v3/projects/59b7ab43f7824f6c8a17988db2a25807` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:32.930537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b7ab43f7824f6c8a17988db2a25807)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:32.932546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:32.982643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b7ab43f7824f6c8a17988db2a25807', 'request_id': 'req-9e840367-c8a8-4bbf-992a-1af03361037e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:32.988947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b7ab43f7824f6c8a17988db2a25807', 'request_id': 'req-9e840367-c8a8-4bbf-992a-1af03361037e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:32.994416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b7ab43f7824f6c8a17988db2a25807', 'request_id': 'req-9e840367-c8a8-4bbf-992a-1af03361037e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:33.008959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] The token cache is being invalidate because project 59b7ab43f7824f6c8a17988db2a25807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:33.009242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:33.009869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e840367-c8a8-4bbf-992a-1af03361037e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:33.011696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1046/2058] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:32 2022] DELETE /identity/v3/projects/59b7ab43f7824f6c8a17988db2a25807 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:33.016787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818a7dc-1d3d-4432-a813-76bad665b642 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.017296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818a7dc-1d3d-4432-a813-76bad665b642 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.017387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818a7dc-1d3d-4432-a813-76bad665b642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.038449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8818a7dc-1d3d-4432-a813-76bad665b642 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:33.039639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8818a7dc-1d3d-4432-a813-76bad665b642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:33.106571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1013/2059] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:33.116007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fa3cb-c83f-4cde-9431-e8d23434f543 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.116568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fa3cb-c83f-4cde-9431-e8d23434f543 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.116568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fa3cb-c83f-4cde-9431-e8d23434f543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.139691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8e1fa3cb-c83f-4cde-9431-e8d23434f543 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:33.145487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8e1fa3cb-c83f-4cde-9431-e8d23434f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:33.163277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1047/2060] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:33.163769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818a7dc-1d3d-4432-a813-76bad665b642 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:33.175244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818a7dc-1d3d-4432-a813-76bad665b642 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:33.180092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:33.180660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.180947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.181224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.181891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:33.182344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:33.184180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:33.198573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1014/2061] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:44:33.203380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fa3cb-c83f-4cde-9431-e8d23434f543 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:33.212799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fa3cb-c83f-4cde-9431-e8d23434f543 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:33.218361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:33.219083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.219568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.219834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.220532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:33.226749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:33.258236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1048/2062] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:33 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:33.263346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:33.278001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb96b01-4b04-4482-bc04-d36a29a76f60 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:33.339952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:33.340574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.340854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.341132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.341765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:33.343729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:33.368110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1015/2063] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:33 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:33.371166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:33.381719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e171ab5-ddf0-4989-bf86-0fb3cc0bbb8e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:33.386939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:33.387585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.387870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.388143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.388772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:33.390394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:33.401233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1049/2064] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:33.406009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:33.418460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e46a8b-b27b-4ae8-8584-e1b47197d472 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:33.423798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70c233-356a-4930-8544-2ec7db6f09d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:33.424413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70c233-356a-4930-8544-2ec7db6f09d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.424695 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70c233-356a-4930-8544-2ec7db6f09d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.424956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70c233-356a-4930-8544-2ec7db6f09d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.425621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70c233-356a-4930-8544-2ec7db6f09d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:33.427286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70c233-356a-4930-8544-2ec7db6f09d2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:33.439626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1016/2065] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:33.443933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:33.457985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee58173-623d-4c91-8ec2-a24ca6254875 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:33.466701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:33.466701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.466701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.466701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] PATH_INFO: `/v3/projects/641b0738db51447b9b816b90900128da/users/00c763381c064af4b52623b0674771ad/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.489271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=641b0738db51447b9b816b90900128da, user_id=00c763381c064af4b52623b0674771ad, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:33.492000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:33.507926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1050/2066] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:33 2022] PUT /identity/v3/projects/641b0738db51447b9b816b90900128da/users/00c763381c064af4b52623b0674771ad/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:33.518897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4efaf-6a15-4e97-b990-3f75b1288cad None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.519263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4efaf-6a15-4e97-b990-3f75b1288cad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.519548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4efaf-6a15-4e97-b990-3f75b1288cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.547652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-23c4efaf-6a15-4e97-b990-3f75b1288cad None None] MFA Rules not processed for user `00c763381c064af4b52623b0674771ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:33.548880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-23c4efaf-6a15-4e97-b990-3f75b1288cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:33.600179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:33.618601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1017/2067] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:33.630693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:33.632514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:33.677470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:33.714757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57cceb-1d74-4424-9e7d-025fdface9ae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:33.721036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:33.721670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.721999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.722218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.731667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:33.733730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:33.745362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1051/2068] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:33.810277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566920f-fe61-4607-864e-8e358a209dc5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.810568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566920f-fe61-4607-864e-8e358a209dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.810871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566920f-fe61-4607-864e-8e358a209dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.838903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a566920f-fe61-4607-864e-8e358a209dc5 None None] MFA Rules not processed for user `814c0227c728493ebd4364bc154cee83`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:33.840947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a566920f-fe61-4607-864e-8e358a209dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:33.923358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1018/2069] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:33.993369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:34.064779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405f47-90f4-4c29-a61c-800860303158 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:34.069623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:34.070418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:34.070704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:34.070982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] PATH_INFO: `/v3/users/00c763381c064af4b52623b0674771ad` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:34.072188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c763381c064af4b52623b0674771ad)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:34.074292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:34.126429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c763381c064af4b52623b0674771ad'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.127212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c763381c064af4b52623b0674771ad'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.134229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c763381c064af4b52623b0674771ad'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.134889 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c763381c064af4b52623b0674771ad'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.144014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c763381c064af4b52623b0674771ad'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.163626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1052/2070] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:33 2022] DELETE /identity/v3/users/00c763381c064af4b52623b0674771ad => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:34.168032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:34.168682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:34.168682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:34.201741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:34.202978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:34.265471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1019/2071] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:34.275256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:34.277054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:34.320975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:34.338273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3921f-8488-4955-a752-2b5c2e4800eb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:34.342919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f909-853a-4c91-b204-6029f547cd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:34.343584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f909-853a-4c91-b204-6029f547cd16 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:34.343871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f909-853a-4c91-b204-6029f547cd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:34.344147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f909-853a-4c91-b204-6029f547cd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:34.354444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f909-853a-4c91-b204-6029f547cd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:34.356019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f909-853a-4c91-b204-6029f547cd16 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:34.367337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1053/2072] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:34.689633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:34.691360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:34.715262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d342e7-ae31-4954-b4ab-932ad9a63de5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:34.723915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:34.724706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:34.725059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:34.725408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] PATH_INFO: `/v3/projects/641b0738db51447b9b816b90900128da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:34.727565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=641b0738db51447b9b816b90900128da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:34.730505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:34.785818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '641b0738db51447b9b816b90900128da', 'request_id': 'req-f67c2d0c-1a29-4653-924c-43a13267b4dc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.791550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '641b0738db51447b9b816b90900128da', 'request_id': 'req-f67c2d0c-1a29-4653-924c-43a13267b4dc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.798213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '641b0738db51447b9b816b90900128da', 'request_id': 'req-f67c2d0c-1a29-4653-924c-43a13267b4dc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.813061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] The token cache is being invalidate because project 641b0738db51447b9b816b90900128da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:34.813360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.813997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f67c2d0c-1a29-4653-924c-43a13267b4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:34.815626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1020/2073] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:34 2022] DELETE /identity/v3/projects/641b0738db51447b9b816b90900128da => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:34.820631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21617a-fa9c-4079-a279-41983fcb19e1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:34.821014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21617a-fa9c-4079-a279-41983fcb19e1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:34.821297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21617a-fa9c-4079-a279-41983fcb19e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:34.842619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3e21617a-fa9c-4079-a279-41983fcb19e1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:34.844480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3e21617a-fa9c-4079-a279-41983fcb19e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:34.921534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1054/2074] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:34.929713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeac74f-6b3d-4da2-b2fe-efab64822b29 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:34.930224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeac74f-6b3d-4da2-b2fe-efab64822b29 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:34.930524 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeac74f-6b3d-4da2-b2fe-efab64822b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:34.954332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-edeac74f-6b3d-4da2-b2fe-efab64822b29 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:34.955654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-edeac74f-6b3d-4da2-b2fe-efab64822b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:34.968812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1021/2075] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:34.970541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21617a-fa9c-4079-a279-41983fcb19e1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:34.982645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21617a-fa9c-4079-a279-41983fcb19e1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:34.987463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:34.988086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:34.988369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:34.988643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:34.989300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:34.989608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:34.991316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.003727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1055/2076] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:35.008272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeac74f-6b3d-4da2-b2fe-efab64822b29 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.018002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeac74f-6b3d-4da2-b2fe-efab64822b29 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.024814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.025560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.025913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.026219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.027036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.028961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.040807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:35.064589 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1022/2077] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:35 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:35.068082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.080948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346db6d-e4ae-414d-b85d-7c85d71b581d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.144004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.144599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.144811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.145083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.145711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.147692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.171714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1056/2078] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:35 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:35.176045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.189091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916b80b-3c5f-4f31-9c12-a3acc08245fd admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.197813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.198539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.198762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.199039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.199782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.202322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.219360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1023/2079] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:35.221431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.234749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de09e1d-5096-4644-8df0-59fd3b625c55 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.240369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69055b7d-1b74-4b76-85c2-3aff129ca8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.241002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69055b7d-1b74-4b76-85c2-3aff129ca8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.241283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69055b7d-1b74-4b76-85c2-3aff129ca8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.241554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69055b7d-1b74-4b76-85c2-3aff129ca8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.242192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69055b7d-1b74-4b76-85c2-3aff129ca8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.243829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69055b7d-1b74-4b76-85c2-3aff129ca8c7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.256770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1057/2080] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:35.261526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.272056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dee85-185c-4eca-83d6-9dca6308f8b0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.278326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.279286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.279575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.280131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] PATH_INFO: `/v3/projects/42e0446037b34e0da84de7a9681b642d/users/7d94c8fb2b7f4366b0c4cb0ceb02ae1d/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.296457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e0446037b34e0da84de7a9681b642d, user_id=7d94c8fb2b7f4366b0c4cb0ceb02ae1d, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.299305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.316952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1024/2081] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:35 2022] PUT /identity/v3/projects/42e0446037b34e0da84de7a9681b642d/users/7d94c8fb2b7f4366b0c4cb0ceb02ae1d/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:35.328911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8b03d-fcea-4ce9-877c-930e734f852b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.329355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8b03d-fcea-4ce9-877c-930e734f852b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.329637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8b03d-fcea-4ce9-877c-930e734f852b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.360098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f1e8b03d-fcea-4ce9-877c-930e734f852b None None] MFA Rules not processed for user `7d94c8fb2b7f4366b0c4cb0ceb02ae1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:35.361883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f1e8b03d-fcea-4ce9-877c-930e734f852b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:35.404116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:35.433039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1058/2082] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:35.438732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.512955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5a0ed-1a1c-437f-872b-6337a7fe006d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.518113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.518732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.519011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.519313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.519948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.521874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.553314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1025/2083] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:35 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:35.557682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8b03d-fcea-4ce9-877c-930e734f852b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.573647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8b03d-fcea-4ce9-877c-930e734f852b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.636242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.636858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.637164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.637441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.638113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.640082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.664704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1059/2084] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:35 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:35.670139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.682234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba398cf8-d7c1-422e-86c5-7bb7a7700123 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.688978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.689980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.690415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.690877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.692035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.694633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.707203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1026/2085] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:35.712294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.724809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd399d8-a172-4074-8f14-a48e26ce6dd8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.730138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.730863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.731157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.731471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] PATH_INFO: `/v3/projects/22edcf23e811469296706bbe9d7c379e/users/738f82fb7fd14466879b35c2a78d0d81/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.748084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] RBAC: Authorizing `identity:create_grant(project_id=22edcf23e811469296706bbe9d7c379e, user_id=738f82fb7fd14466879b35c2a78d0d81, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.750810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.764626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1060/2086] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:35 2022] PUT /identity/v3/projects/22edcf23e811469296706bbe9d7c379e/users/738f82fb7fd14466879b35c2a78d0d81/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:35.768895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.779543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5d5c5-7c8f-4859-94e1-69d14365893f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.842535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f8db-e545-48f8-9b4a-5ed4e3e1af07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.843158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8db-e545-48f8-9b4a-5ed4e3e1af07 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.843595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8db-e545-48f8-9b4a-5ed4e3e1af07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.843873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f8db-e545-48f8-9b4a-5ed4e3e1af07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.844536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7f8db-e545-48f8-9b4a-5ed4e3e1af07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.846473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7f8db-e545-48f8-9b4a-5ed4e3e1af07 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.861331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1027/2087] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:35.866097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.878712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb3789-e7c8-4333-acd8-26055ebab52f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.884449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.885052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.885583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.885887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] PATH_INFO: `/v3/projects/22edcf23e811469296706bbe9d7c379e/users/738f82fb7fd14466879b35c2a78d0d81/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.888020 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=22edcf23e811469296706bbe9d7c379e, user_id=738f82fb7fd14466879b35c2a78d0d81, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.890873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.901658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1061/2088] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:35 2022] PUT /identity/v3/projects/22edcf23e811469296706bbe9d7c379e/users/738f82fb7fd14466879b35c2a78d0d81/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:35.911964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23f808-f541-4a6b-b3ec-770ddb80f928 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.912256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23f808-f541-4a6b-b3ec-770ddb80f928 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.912538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23f808-f541-4a6b-b3ec-770ddb80f928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.941692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ee23f808-f541-4a6b-b3ec-770ddb80f928 None None] MFA Rules not processed for user `738f82fb7fd14466879b35c2a78d0d81`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:35.942661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ee23f808-f541-4a6b-b3ec-770ddb80f928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:35.956476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:36.020672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1028/2089] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:36.033382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:36.035236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:36.099955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a625e-aa6d-477b-b58f-e69dd383eaae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:36.104562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504e25a-6b2a-482f-82b3-4b41ec995928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:36.105230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e25a-6b2a-482f-82b3-4b41ec995928 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:36.105512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e25a-6b2a-482f-82b3-4b41ec995928 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:36.105789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e25a-6b2a-482f-82b3-4b41ec995928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:36.117640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504e25a-6b2a-482f-82b3-4b41ec995928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:36.118978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504e25a-6b2a-482f-82b3-4b41ec995928 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:36.129915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1062/2090] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:36.224654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23f808-f541-4a6b-b3ec-770ddb80f928 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:36.235886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23f808-f541-4a6b-b3ec-770ddb80f928 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:36.239673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:36.240324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:36.240609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:36.240886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:36.300794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:36.302454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:36.313438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1029/2091] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:46.124984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:46.308327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=50 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:54.222080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c54ba-413e-4e55-974e-f109096a97f5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:54.222627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c54ba-413e-4e55-974e-f109096a97f5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:54.222627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c54ba-413e-4e55-974e-f109096a97f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:54.243240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d85c54ba-413e-4e55-974e-f109096a97f5 None None] MFA Rules not processed for user `814c0227c728493ebd4364bc154cee83`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:54.244641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d85c54ba-413e-4e55-974e-f109096a97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:54.303963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1063/2092] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:54.747792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:54.752073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:54.856793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7743d1-e5d3-47bf-9bf5-34c3b9fd598b service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:54.863161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:54.863931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:54.864268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:54.864812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] PATH_INFO: `/v3/users/b2d41acfd6474e78929fd674a59233a9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:54.866562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d41acfd6474e78929fd674a59233a9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:54.868984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:54.942116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d41acfd6474e78929fd674a59233a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:54.953175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d41acfd6474e78929fd674a59233a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:54.964121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d41acfd6474e78929fd674a59233a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:54.965908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d41acfd6474e78929fd674a59233a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:54.966624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d41acfd6474e78929fd674a59233a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:54.991419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1030/2093] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:54 2022] DELETE /identity/v3/users/b2d41acfd6474e78929fd674a59233a9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:55.008433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.008433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.008433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.016143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:55.017380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:55.080377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1064/2094] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:55.091145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:55.092675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:55.152341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb75e4-4837-414f-b48a-9432c6083c41 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:55.156707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be70f40-2e80-4985-af68-d2bb4556f04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:55.157685 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be70f40-2e80-4985-af68-d2bb4556f04d service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.158218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be70f40-2e80-4985-af68-d2bb4556f04d service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.158590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be70f40-2e80-4985-af68-d2bb4556f04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.170356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be70f40-2e80-4985-af68-d2bb4556f04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:55.172014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be70f40-2e80-4985-af68-d2bb4556f04d service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:55.182690 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1031/2095] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:55.490337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:55.491876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:55.505306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20345053-13ef-4195-99a5-1cbc9590681c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:55.525367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:55.526068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.526772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.527066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] PATH_INFO: `/v3/projects/09db79d382e149acbaac327f95ae252d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.529494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=09db79d382e149acbaac327f95ae252d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:55.531845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:55.583626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09db79d382e149acbaac327f95ae252d', 'request_id': 'req-8e022504-189d-4b53-98e7-4edd121de3cd'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:55.592171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09db79d382e149acbaac327f95ae252d', 'request_id': 'req-8e022504-189d-4b53-98e7-4edd121de3cd'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:55.598393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09db79d382e149acbaac327f95ae252d', 'request_id': 'req-8e022504-189d-4b53-98e7-4edd121de3cd'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:55.612246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] The token cache is being invalidate because project 09db79d382e149acbaac327f95ae252d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:55.612533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:55.613356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e022504-189d-4b53-98e7-4edd121de3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:55.615145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1065/2096] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:55 2022] DELETE /identity/v3/projects/09db79d382e149acbaac327f95ae252d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:55.621146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269861-b348-45a3-bfee-f4bee4b54fea None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.621416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269861-b348-45a3-bfee-f4bee4b54fea None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.621683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269861-b348-45a3-bfee-f4bee4b54fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.642522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-af269861-b348-45a3-bfee-f4bee4b54fea None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:55.643764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-af269861-b348-45a3-bfee-f4bee4b54fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:55.662764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:55.715256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1032/2097] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:55.725159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0d5a9-712d-49d3-b4f3-2abaf57a842a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.725553 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0d5a9-712d-49d3-b4f3-2abaf57a842a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.726166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0d5a9-712d-49d3-b4f3-2abaf57a842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.747829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bcd0d5a9-712d-49d3-b4f3-2abaf57a842a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:55.749046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bcd0d5a9-712d-49d3-b4f3-2abaf57a842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:55.757555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1066/2098] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:55.762169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269861-b348-45a3-bfee-f4bee4b54fea None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:55.772598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269861-b348-45a3-bfee-f4bee4b54fea None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:55.778777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:55.779436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.779703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.779988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.780644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:55.780945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:55.782971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:55.795594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1033/2099] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:55.800047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0d5a9-712d-49d3-b4f3-2abaf57a842a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:55.809940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0d5a9-712d-49d3-b4f3-2abaf57a842a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:55.815685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:55.816295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.816578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.816851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.817487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:55.819660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:55.845290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:55.857802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1067/2100] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:55 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:55.858936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:55.869276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f59169-0463-4278-ad4e-72e1e7e7cc47 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:55.952159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:55.952857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:55.955872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:55.957123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:55.958680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:55.963890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:56.003264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1034/2101] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:55 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:56.006716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:56.028722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2f7b1-43e2-4091-8e9d-1c9a35618382 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:56.049372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:56.049372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:56.049721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:56.049881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:56.050790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:56.052900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:56.076280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1068/2102] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:44:56.080001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:56.096482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ddb2a-2f8e-44fb-8424-f023b0bac796 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:56.101900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4bbb5-9691-479e-9c98-439756e3a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:56.102519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4bbb5-9691-479e-9c98-439756e3a149 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:56.102799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4bbb5-9691-479e-9c98-439756e3a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:56.103250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4bbb5-9691-479e-9c98-439756e3a149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:56.104008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4bbb5-9691-479e-9c98-439756e3a149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:56.105818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4bbb5-9691-479e-9c98-439756e3a149 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:56.133132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1035/2103] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:56.137279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:56.148559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04de39d-45a0-4207-916c-fc9f1c2ef439 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:56.164267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:56.170812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:56.171726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:56.172348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] PATH_INFO: `/v3/projects/0bf38c354bea44f5a503e76fa53b20f6/users/7957b80bf6e843d4a7a0b11f13a80db8/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:56.211738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf38c354bea44f5a503e76fa53b20f6, user_id=7957b80bf6e843d4a7a0b11f13a80db8, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:56.220369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:56.253082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1069/2104] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:56 2022] PUT /identity/v3/projects/0bf38c354bea44f5a503e76fa53b20f6/users/7957b80bf6e843d4a7a0b11f13a80db8/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:56.259562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5d5d-a991-40f9-9ee7-7561178d458b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:56.259857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5d5d-a991-40f9-9ee7-7561178d458b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:56.260136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5d5d-a991-40f9-9ee7-7561178d458b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:56.294233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d76e5d5d-a991-40f9-9ee7-7561178d458b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:56.295593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d76e5d5d-a991-40f9-9ee7-7561178d458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:56.375861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1036/2105] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:56.386333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:56.388719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:56.454875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de94213-7eb1-4536-84a6-e0b21af2e842 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:56.458898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:56.459637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:56.459921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:56.460210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:56.470439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:56.474984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:56.482969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1070/2106] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:01.088975 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563eb76-2729-494c-8e6e-797cb84db485 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:01.089505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563eb76-2729-494c-8e6e-797cb84db485 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:01.089505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563eb76-2729-494c-8e6e-797cb84db485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:01.119790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c563eb76-2729-494c-8e6e-797cb84db485 None None] MFA Rules not processed for user `7957b80bf6e843d4a7a0b11f13a80db8`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:01.121020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c563eb76-2729-494c-8e6e-797cb84db485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:01.139413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:01.211720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1037/2107] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:01.215929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:01.235706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139a3a6-8b05-4196-9f9b-25aef7add287 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:01.241494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:01.242146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:01.242532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:01.242815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:01.243484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:01.245475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:01.294692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1071/2108] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:45:01 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:01.298663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563eb76-2729-494c-8e6e-797cb84db485 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:01.315396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563eb76-2729-494c-8e6e-797cb84db485 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:01.390188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:01.391474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:01.391878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:01.392185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:01.392869 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:01.395176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:01.423711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1038/2109] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:45:01 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:45:01.431348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:01.448468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e5c87-1f87-4cb2-a20c-eb9aaa68120c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:01.456560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:01.457408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:01.457843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:01.458265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:01.459122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:01.460816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:01.477592 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1072/2110] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:45:01.478931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:01.489775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a7c2f-96c3-428b-b805-12b5ab583dc1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:01.498161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:01.498865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:01.499148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:01.499468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] PATH_INFO: `/v3/projects/3a06b7cea7d34413b2f95c1db2583af0/users/bd5127b5d0034af2a4bac2e0330a400c/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:01.516614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a06b7cea7d34413b2f95c1db2583af0, user_id=bd5127b5d0034af2a4bac2e0330a400c, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:01.519312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:01.534894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1039/2111] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:01 2022] PUT /identity/v3/projects/3a06b7cea7d34413b2f95c1db2583af0/users/bd5127b5d0034af2a4bac2e0330a400c/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:01.538957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:01.549859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e0bc9-b737-4df3-a13d-a31fea6bdca3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:01.613443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbe80d-4cb9-4d43-b0ae-4f1e9691b02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:01.614289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe80d-4cb9-4d43-b0ae-4f1e9691b02e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:01.614814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe80d-4cb9-4d43-b0ae-4f1e9691b02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:01.615344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe80d-4cb9-4d43-b0ae-4f1e9691b02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:01.616220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbe80d-4cb9-4d43-b0ae-4f1e9691b02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:01.617828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbe80d-4cb9-4d43-b0ae-4f1e9691b02e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:01.629615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1073/2112] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:01.633844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:01.644993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0867a1-4e98-4e4d-b266-ced4d63898b1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:01.649727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:01.650399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:01.650684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:01.650984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] PATH_INFO: `/v3/projects/3a06b7cea7d34413b2f95c1db2583af0/users/bd5127b5d0034af2a4bac2e0330a400c/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:01.654458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a06b7cea7d34413b2f95c1db2583af0, user_id=bd5127b5d0034af2a4bac2e0330a400c, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:01.657171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:01.672800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1040/2113] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:01 2022] PUT /identity/v3/projects/3a06b7cea7d34413b2f95c1db2583af0/users/bd5127b5d0034af2a4bac2e0330a400c/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:06.742366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d241e25-8577-4eb2-85fe-3a87ba106194 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:06.742945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d241e25-8577-4eb2-85fe-3a87ba106194 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:06.743062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d241e25-8577-4eb2-85fe-3a87ba106194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:06.781990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1d241e25-8577-4eb2-85fe-3a87ba106194 None None] MFA Rules not processed for user `bd5127b5d0034af2a4bac2e0330a400c`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:06.784091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1d241e25-8577-4eb2-85fe-3a87ba106194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:06.805177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:06.861381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1074/2114] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:06.873930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:06.876274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:06.981477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9a5d2-6f37-45a9-84de-0d674fbafaa4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:06.988760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:06.989494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:06.989709 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:06.989982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:06.997583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:07.082468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:07.084903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:07.098921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1041/2115] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:08.961395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d835351-615e-495a-8cf0-535c4226899c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:08.961878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d835351-615e-495a-8cf0-535c4226899c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:08.961958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d835351-615e-495a-8cf0-535c4226899c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:08.987229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4d835351-615e-495a-8cf0-535c4226899c None None] MFA Rules not processed for user `7957b80bf6e843d4a7a0b11f13a80db8`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:08.988468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4d835351-615e-495a-8cf0-535c4226899c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:08.995784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1075/2116] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:17.093088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=24 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:18.981729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=23 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:19.828737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:19.830263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:19.909356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed393b-47c1-4580-a0f5-bdf5cf211d06 service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:19.916994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:19.917878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:19.918259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:19.918791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] PATH_INFO: `/v3/users/7d94c8fb2b7f4366b0c4cb0ceb02ae1d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:19.920267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d94c8fb2b7f4366b0c4cb0ceb02ae1d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:19.922640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:20.000227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d94c8fb2b7f4366b0c4cb0ceb02ae1d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.012645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d94c8fb2b7f4366b0c4cb0ceb02ae1d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.023086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d94c8fb2b7f4366b0c4cb0ceb02ae1d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.024666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d94c8fb2b7f4366b0c4cb0ceb02ae1d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.025348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d94c8fb2b7f4366b0c4cb0ceb02ae1d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.050423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1042/2117] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:45:19 2022] DELETE /identity/v3/users/7d94c8fb2b7f4366b0c4cb0ceb02ae1d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:20.054821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:20.055322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:20.055584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:20.079015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:20.080491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:20.189173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1076/2118] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:20.200395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:20.202413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:20.276080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da291f7-7d43-4b65-8b4b-9fd6b6fbe8e0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:20.280324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:20.281095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:20.281490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:20.281808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:20.292689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:20.294740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:20.314522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1043/2119] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:20.615859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:20.617385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:20.630692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97747349-d769-4956-bd98-ee1d1d1c143b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:20.641431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:20.642131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:20.642432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:20.642725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] PATH_INFO: `/v3/projects/42e0446037b34e0da84de7a9681b642d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:20.643910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e0446037b34e0da84de7a9681b642d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:20.646147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:20.696959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e0446037b34e0da84de7a9681b642d', 'request_id': 'req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.703167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e0446037b34e0da84de7a9681b642d', 'request_id': 'req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.708823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e0446037b34e0da84de7a9681b642d', 'request_id': 'req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.723489 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] The token cache is being invalidate because project 42e0446037b34e0da84de7a9681b642d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:45:20.723814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.724567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.726088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1077/2120] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:45:20 2022] DELETE /identity/v3/projects/42e0446037b34e0da84de7a9681b642d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:20.730230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:20.741156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4f578-80b0-4836-ad5b-3ec44032c2bb service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:20.764734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:20.806468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:20.807122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:20.807448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:20.807727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] PATH_INFO: `/v3/users/738f82fb7fd14466879b35c2a78d0d81` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:20.808887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] RBAC: Authorizing `identity:delete_user(user_id=738f82fb7fd14466879b35c2a78d0d81)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:20.810854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:20.861437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738f82fb7fd14466879b35c2a78d0d81'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.869533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738f82fb7fd14466879b35c2a78d0d81'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.879203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738f82fb7fd14466879b35c2a78d0d81'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.880031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738f82fb7fd14466879b35c2a78d0d81'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.880716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09f52a4e-a113-44be-b6a9-4590232d999a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738f82fb7fd14466879b35c2a78d0d81'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:20.900190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1044/2121] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:45:20 2022] DELETE /identity/v3/users/738f82fb7fd14466879b35c2a78d0d81 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:45:21.245718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:21.248922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:21.290720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7535c55-b6cd-46a8-a74d-6ffa876b61f7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:21.355731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:21.356379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.356664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.356939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] PATH_INFO: `/v3/projects/22edcf23e811469296706bbe9d7c379e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.358065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22edcf23e811469296706bbe9d7c379e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:21.360089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:21.392831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:21.410724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22edcf23e811469296706bbe9d7c379e', 'request_id': 'req-382bea9c-44c8-4dfd-b29e-16a3571cddf1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:21.417253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22edcf23e811469296706bbe9d7c379e', 'request_id': 'req-382bea9c-44c8-4dfd-b29e-16a3571cddf1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:21.423653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22edcf23e811469296706bbe9d7c379e', 'request_id': 'req-382bea9c-44c8-4dfd-b29e-16a3571cddf1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:21.438209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] The token cache is being invalidate because project 22edcf23e811469296706bbe9d7c379e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:45:21.438365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:21.439243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-382bea9c-44c8-4dfd-b29e-16a3571cddf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:21.440787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1078/2122] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:45:21 2022] DELETE /identity/v3/projects/22edcf23e811469296706bbe9d7c379e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:21.446219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0de9c-7b4c-4720-9f83-646534e4b988 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.446529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0de9c-7b4c-4720-9f83-646534e4b988 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.446872 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0de9c-7b4c-4720-9f83-646534e4b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.468500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f8a0de9c-7b4c-4720-9f83-646534e4b988 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:21.469966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f8a0de9c-7b4c-4720-9f83-646534e4b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:21.530231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1045/2123] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:21.538417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4da7de-daf2-485f-b897-6cad4b205668 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.538705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4da7de-daf2-485f-b897-6cad4b205668 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.539027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4da7de-daf2-485f-b897-6cad4b205668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.559597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2c4da7de-daf2-485f-b897-6cad4b205668 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:21.560843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2c4da7de-daf2-485f-b897-6cad4b205668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:21.570391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1079/2124] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:21.574887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0de9c-7b4c-4720-9f83-646534e4b988 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:21.585058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0de9c-7b4c-4720-9f83-646534e4b988 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:21.590429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:21.591041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.591367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.591646 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.592297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:45:21.592606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:21.594231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:21.606814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1046/2125] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:45:21.613049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4da7de-daf2-485f-b897-6cad4b205668 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:21.622695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4da7de-daf2-485f-b897-6cad4b205668 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:21.628645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:21.629296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.629575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.629843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.630481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:21.632442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:21.663785 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1080/2126] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:45:21 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:45:21.668200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:21.678619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ad774-a383-4de0-a18b-c8afb453f919 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:21.705593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:21.745994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:21.746848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.747304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.747586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.748225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:21.750501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:21.776542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1047/2127] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:45:21 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:21.777949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:21.789323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097453d-41ad-4173-aceb-52caeb7d3e28 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:21.794295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:21.794938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.795243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.795526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.796217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:21.798037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:21.808616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1081/2128] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:21.813112 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:21.823725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df916f-1260-4d5b-9344-2c919603c6ba admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:21.828714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7b8d0-d57b-4a64-90fd-bc06d6adb1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:21.829304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b8d0-d57b-4a64-90fd-bc06d6adb1db admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.829594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b8d0-d57b-4a64-90fd-bc06d6adb1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.829868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7b8d0-d57b-4a64-90fd-bc06d6adb1db admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.830522 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b8d0-d57b-4a64-90fd-bc06d6adb1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:21.832162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7b8d0-d57b-4a64-90fd-bc06d6adb1db admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:21.842452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1048/2129] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:21.847372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:21.864436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea3f77-fe45-416b-98c2-eac69c4429ff admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:21.873615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:21.874887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.874887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.875112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] PATH_INFO: `/v3/projects/be11feb1e7c145b6a9fef291f7ef6c16/users/18feea9da1ee40e19c45100591d44410/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.890517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be11feb1e7c145b6a9fef291f7ef6c16, user_id=18feea9da1ee40e19c45100591d44410, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:21.892885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:21.908891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1082/2130] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:21 2022] PUT /identity/v3/projects/be11feb1e7c145b6a9fef291f7ef6c16/users/18feea9da1ee40e19c45100591d44410/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:21.920485 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420e5c-a14a-4727-9be1-e1c1f32a61eb None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:21.921040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420e5c-a14a-4727-9be1-e1c1f32a61eb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:21.921040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57420e5c-a14a-4727-9be1-e1c1f32a61eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:21.955076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-57420e5c-a14a-4727-9be1-e1c1f32a61eb None None] MFA Rules not processed for user `18feea9da1ee40e19c45100591d44410`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:21.956557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-57420e5c-a14a-4727-9be1-e1c1f32a61eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:22.026221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1049/2131] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:22.031179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.104580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188725b6-c143-4967-a7dd-13cb48aa0274 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.111738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.112359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.112658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.112917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.113554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.115497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.152435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1083/2132] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:45:22 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:22.156921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420e5c-a14a-4727-9be1-e1c1f32a61eb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.168192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57420e5c-a14a-4727-9be1-e1c1f32a61eb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.231168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.231822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.232103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.232378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.233012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.235831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.258753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1050/2133] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:45:22 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:22.262933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.273725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9f633-ee9f-463c-8544-73ceff74a16f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.278389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.278995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.279309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.279583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.280371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.281978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.292886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1084/2134] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:22.297227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.307580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520ea84-40e3-4968-9a9a-b6c74148c8e7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.319673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.320353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.320658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.320937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] PATH_INFO: `/v3/projects/285cba5954f84bb1b2fbcf7c182f8a1f/users/4c4fe2e6b35848b4aae1919c7f258576/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.338042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] RBAC: Authorizing `identity:create_grant(project_id=285cba5954f84bb1b2fbcf7c182f8a1f, user_id=4c4fe2e6b35848b4aae1919c7f258576, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.341044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.358986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1051/2135] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:22 2022] PUT /identity/v3/projects/285cba5954f84bb1b2fbcf7c182f8a1f/users/4c4fe2e6b35848b4aae1919c7f258576/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:22.374193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.385561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b04b31-0a47-43ea-9681-b074f0902aae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.415556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:22.457003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061ef2-e12d-4a36-af09-078c30b5c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.457634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061ef2-e12d-4a36-af09-078c30b5c9db admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.457922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061ef2-e12d-4a36-af09-078c30b5c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.458205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061ef2-e12d-4a36-af09-078c30b5c9db admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.458878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061ef2-e12d-4a36-af09-078c30b5c9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.460569 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061ef2-e12d-4a36-af09-078c30b5c9db admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.474530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1085/2136] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:22.476421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.483021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:22.489798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870c102-f07e-4f32-97dc-811d7d87665e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.495028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.495724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.496029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.496364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] PATH_INFO: `/v3/projects/285cba5954f84bb1b2fbcf7c182f8a1f/users/4c4fe2e6b35848b4aae1919c7f258576/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.498266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285cba5954f84bb1b2fbcf7c182f8a1f, user_id=4c4fe2e6b35848b4aae1919c7f258576, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.501411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.512661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1052/2137] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:22 2022] PUT /identity/v3/projects/285cba5954f84bb1b2fbcf7c182f8a1f/users/4c4fe2e6b35848b4aae1919c7f258576/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:22.524016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705359-1527-45f2-8b6b-16c1edf59932 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.524303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705359-1527-45f2-8b6b-16c1edf59932 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.524582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705359-1527-45f2-8b6b-16c1edf59932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.553749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7c705359-1527-45f2-8b6b-16c1edf59932 None None] MFA Rules not processed for user `4c4fe2e6b35848b4aae1919c7f258576`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:22.554894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7c705359-1527-45f2-8b6b-16c1edf59932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:22.635726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1086/2138] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:45:22.647675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.649595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:22.712327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f98032-2813-45b6-8d1d-efd20e279590 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.717119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522f4f4-a7e5-4f59-881a-2b6f822185b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.717826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522f4f4-a7e5-4f59-881a-2b6f822185b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.718118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522f4f4-a7e5-4f59-881a-2b6f822185b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.718449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522f4f4-a7e5-4f59-881a-2b6f822185b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.728712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522f4f4-a7e5-4f59-881a-2b6f822185b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.730186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522f4f4-a7e5-4f59-881a-2b6f822185b4 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.740069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1053/2139] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:22.843137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705359-1527-45f2-8b6b-16c1edf59932 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:22.856677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705359-1527-45f2-8b6b-16c1edf59932 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:22.861689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:22.862312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:22.862594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:22.862870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:22.935421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:22.937533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:22.947783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1087/2140] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:32.735559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:32.943433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=80 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:39.135239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cbbfa-61ad-4fab-bea4-7ea3e6f128ac None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:39.135978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cbbfa-61ad-4fab-bea4-7ea3e6f128ac None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:39.136292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cbbfa-61ad-4fab-bea4-7ea3e6f128ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:39.155874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-749cbbfa-61ad-4fab-bea4-7ea3e6f128ac None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:39.157087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-749cbbfa-61ad-4fab-bea4-7ea3e6f128ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:39.215600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1054/2141] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:45:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:39.259098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:39.260735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:39.343979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518ff64-bad5-47c8-a0d4-e7850a511469 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:39.349597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:39.350222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:39.350498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:39.350772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:39.360861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:39.362439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:39.372590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1088/2142] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:45:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:45:41.241973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f281d-783f-4257-9ce4-2b0e98aba599 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:41.243349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f281d-783f-4257-9ce4-2b0e98aba599 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:41.243966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f281d-783f-4257-9ce4-2b0e98aba599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:41.280878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ea0f281d-783f-4257-9ce4-2b0e98aba599 None None] MFA Rules not processed for user `f264e354671d4456bb33668fb11d9636`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:41.282288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ea0f281d-783f-4257-9ce4-2b0e98aba599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:41.384196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1055/2143] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:41.991837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:42.152729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b3c0f-7e14-41ba-aab3-fd0d8220e21d service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:42.164768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:42.166036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:42.167436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:42.168083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] PATH_INFO: `/v3/users/18feea9da1ee40e19c45100591d44410` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:42.169931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] RBAC: Authorizing `identity:delete_user(user_id=18feea9da1ee40e19c45100591d44410)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:42.174222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:42.239890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18feea9da1ee40e19c45100591d44410'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:42.242867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18feea9da1ee40e19c45100591d44410'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:42.251763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18feea9da1ee40e19c45100591d44410'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:42.253404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18feea9da1ee40e19c45100591d44410'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:42.274542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18feea9da1ee40e19c45100591d44410'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:42.316792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:42.317656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:42.318115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:42.320951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1089/2144] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:45:41 2022] DELETE /identity/v3/users/18feea9da1ee40e19c45100591d44410 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:42.347889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:42.349443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:42.422674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1056/2145] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:42.433926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:42.435740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:42.453759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:42.571337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34cf8-7024-4fd2-b684-1d4b4123988d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:42.575820 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:42.577138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:42.577726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:42.578432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:42.595514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:42.596907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:42.611345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1090/2146] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:45:42.993724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:42.995802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:43.033656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc641b-5c14-4dff-90f2-b2ae7a314d04 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:43.052714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:43.053840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:43.054549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:43.055070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] PATH_INFO: `/v3/projects/be11feb1e7c145b6a9fef291f7ef6c16` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:43.057055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be11feb1e7c145b6a9fef291f7ef6c16)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:43.062956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:43.138224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:43.141731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be11feb1e7c145b6a9fef291f7ef6c16', 'request_id': 'req-9a5fedbb-31a6-48fa-85b4-212b17123a03'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.156201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be11feb1e7c145b6a9fef291f7ef6c16', 'request_id': 'req-9a5fedbb-31a6-48fa-85b4-212b17123a03'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.171004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be11feb1e7c145b6a9fef291f7ef6c16', 'request_id': 'req-9a5fedbb-31a6-48fa-85b4-212b17123a03'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.210334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] The token cache is being invalidate because project be11feb1e7c145b6a9fef291f7ef6c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:45:43.210911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.212041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.214417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1057/2147] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:45:42 2022] DELETE /identity/v3/projects/be11feb1e7c145b6a9fef291f7ef6c16 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:43.218953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:43.232872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4777c56-85df-4813-8895-de0df5950ddc service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:43.349787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:43.350517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:43.350903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:43.351176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] PATH_INFO: `/v3/users/4c4fe2e6b35848b4aae1919c7f258576` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:43.352602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c4fe2e6b35848b4aae1919c7f258576)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:43.354764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:43.430605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c4fe2e6b35848b4aae1919c7f258576'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.431437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c4fe2e6b35848b4aae1919c7f258576'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.446173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c4fe2e6b35848b4aae1919c7f258576'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.447925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c4fe2e6b35848b4aae1919c7f258576'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.467997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2d74fa69-8883-4e42-8ffe-fa19b0d84f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c4fe2e6b35848b4aae1919c7f258576'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:43.494834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1091/2148] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:45:43 2022] DELETE /identity/v3/users/4c4fe2e6b35848b4aae1919c7f258576 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:44.093710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:44.095758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:44.149647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fedbb-31a6-48fa-85b4-212b17123a03 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:44.282454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:44.283240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.284114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.284114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] PATH_INFO: `/v3/projects/285cba5954f84bb1b2fbcf7c182f8a1f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.285498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=285cba5954f84bb1b2fbcf7c182f8a1f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:44.287687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:44.348038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285cba5954f84bb1b2fbcf7c182f8a1f', 'request_id': 'req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:44.356164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285cba5954f84bb1b2fbcf7c182f8a1f', 'request_id': 'req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:44.365235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285cba5954f84bb1b2fbcf7c182f8a1f', 'request_id': 'req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:44.383674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] The token cache is being invalidate because project 285cba5954f84bb1b2fbcf7c182f8a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:45:44.395777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:44.401371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e72b784c-7f64-4d10-bad0-3982a9a5fe8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:44.403479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1058/2149] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:45:44 2022] DELETE /identity/v3/projects/285cba5954f84bb1b2fbcf7c182f8a1f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:44.408372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bea56-ab28-4328-8ffc-3b5dfc586612 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.408933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bea56-ab28-4328-8ffc-3b5dfc586612 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.409129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bea56-ab28-4328-8ffc-3b5dfc586612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.446488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5f4bea56-ab28-4328-8ffc-3b5dfc586612 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:44.448090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5f4bea56-ab28-4328-8ffc-3b5dfc586612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:44.495237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:44.553317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1092/2150] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:44.562246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aff49-e041-42e4-ba01-642607d5c876 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.562580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aff49-e041-42e4-ba01-642607d5c876 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.562826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aff49-e041-42e4-ba01-642607d5c876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.589701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-123aff49-e041-42e4-ba01-642607d5c876 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:44.590977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-123aff49-e041-42e4-ba01-642607d5c876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:44.599732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1059/2151] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:44.604133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bea56-ab28-4328-8ffc-3b5dfc586612 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:44.615521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bea56-ab28-4328-8ffc-3b5dfc586612 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:44.620989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:44.621616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.621933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.622214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.622965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:45:44.623311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:44.624992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:44.639065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1093/2152] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:45:44.643232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aff49-e041-42e4-ba01-642607d5c876 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:44.653198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aff49-e041-42e4-ba01-642607d5c876 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:44.657992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:44.658636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.658915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.659216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.659895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:44.661865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:44.701602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1060/2153] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:45:44 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:44.705715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:44.721690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee8333-6b04-40ba-ae58-86c58878c4e7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:44.836452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:44.837214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.837594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.838093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.839336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:44.841525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:44.899636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1094/2154] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:45:44 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:44.903819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:44.920256 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4c715-2a45-4fe4-99d7-ea3dc41a8797 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:44.944393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:44.944795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.945088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.945367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.946095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:44.954892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:44.972195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1061/2155] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:45:44.973497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:44.985530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caee5a-e128-4e0c-853b-98fbb6ec42eb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:44.992071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b22d2-3ec9-44ef-ae47-25bf6afebc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:44.992702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b22d2-3ec9-44ef-ae47-25bf6afebc4f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:44.994782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b22d2-3ec9-44ef-ae47-25bf6afebc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:44.995087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b22d2-3ec9-44ef-ae47-25bf6afebc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:44.996151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b22d2-3ec9-44ef-ae47-25bf6afebc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:44.997607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b22d2-3ec9-44ef-ae47-25bf6afebc4f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:45.015349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1095/2156] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:45.019008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:45.043153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10737f3c-dfde-4d5c-aa58-75e6951d6878 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:45.061736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:45.062708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:45.063127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:45.063539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] PATH_INFO: `/v3/projects/7139329fd50d4910820437fe17d09a1a/users/9acdb666b2a044bca60c01564fc050a5/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:45.104710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7139329fd50d4910820437fe17d09a1a, user_id=9acdb666b2a044bca60c01564fc050a5, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:45.116179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:45.150560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:45.172689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1062/2157] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:45 2022] PUT /identity/v3/projects/7139329fd50d4910820437fe17d09a1a/users/9acdb666b2a044bca60c01564fc050a5/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:45.172925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ed2c-a4fd-4acb-8af9-3ad181e672ec None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:45.173317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ed2c-a4fd-4acb-8af9-3ad181e672ec None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:45.173782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ed2c-a4fd-4acb-8af9-3ad181e672ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:45.206236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f6a1ed2c-a4fd-4acb-8af9-3ad181e672ec None None] MFA Rules not processed for user `9acdb666b2a044bca60c01564fc050a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:45.207520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f6a1ed2c-a4fd-4acb-8af9-3ad181e672ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:45.319180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1096/2158] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:45.324991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:45.465746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc3bcc-14f7-4ac2-8505-ecd15fa0d07e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:45.474438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:45.477626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:45.478206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:45.478999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:45.480255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:45.485672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:45.540737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1063/2159] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:45:45 2022] POST /identity/v3/projects => generated 369 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:45.546459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1ed2c-a4fd-4acb-8af9-3ad181e672ec None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:45.565537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1ed2c-a4fd-4acb-8af9-3ad181e672ec None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:45.616050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:45.682134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:45.683211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:45.683482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:45.683916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:45.690675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:45.693573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:45.744319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1097/2160] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:45:45 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:45.745677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:45.767900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa68de2-1554-4c44-a2d7-cecc471a712f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:45.788002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:45.789750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:45.790204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:45.790487 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:45.791226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:45.798389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:45.824153 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1064/2161] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:45.825644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:45.841393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2b4b9-d1ed-4932-a919-56612d047f82 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:45.855436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:45.856416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:45.858164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:45.858457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] PATH_INFO: `/v3/projects/949bf53270bc46bdace845512886eadb/users/4a2de621c3934549a432ebc68e3c2941/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:45.890646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949bf53270bc46bdace845512886eadb, user_id=4a2de621c3934549a432ebc68e3c2941, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:45.893428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:45.933575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1098/2162] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:45 2022] PUT /identity/v3/projects/949bf53270bc46bdace845512886eadb/users/4a2de621c3934549a432ebc68e3c2941/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:45:45.937967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:45.953418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9d805-561a-4760-922a-3a9816214c99 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:46.143907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cb46e-86f4-444a-b75b-347c8c06e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:46.144545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb46e-86f4-444a-b75b-347c8c06e3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:46.144756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb46e-86f4-444a-b75b-347c8c06e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:46.152843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb46e-86f4-444a-b75b-347c8c06e3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:46.153674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cb46e-86f4-444a-b75b-347c8c06e3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:46.155378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cb46e-86f4-444a-b75b-347c8c06e3d7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:46.221199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:46.227553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1065/2163] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:46.236771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e7e03-bdb1-46eb-a022-f1f18cc6a001 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:46.255234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:46.260197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:46.261835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:46.262351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] PATH_INFO: `/v3/projects/949bf53270bc46bdace845512886eadb/users/4a2de621c3934549a432ebc68e3c2941/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:46.285975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949bf53270bc46bdace845512886eadb, user_id=4a2de621c3934549a432ebc68e3c2941, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:46.293382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:46.310321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1099/2164] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:46 2022] PUT /identity/v3/projects/949bf53270bc46bdace845512886eadb/users/4a2de621c3934549a432ebc68e3c2941/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:45:46.322687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f715183-a591-40ca-bc73-32b500f771e7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:46.322853 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f715183-a591-40ca-bc73-32b500f771e7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:46.323455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f715183-a591-40ca-bc73-32b500f771e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:46.385843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4f715183-a591-40ca-bc73-32b500f771e7 None None] MFA Rules not processed for user `4a2de621c3934549a432ebc68e3c2941`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:46.390660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4f715183-a591-40ca-bc73-32b500f771e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:46.657409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:46.712070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:46.714938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1066/2165] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:46.716503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:46.867676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4987d1-3bd7-4cdc-a1be-49454ac22593 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:46.883961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:46.884795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:46.885088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:46.885366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:46.920519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:46.923757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:46.970232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1100/2166] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:48.736338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229d38-1526-43dc-b0c4-f52ae7b32175 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:48.737830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229d38-1526-43dc-b0c4-f52ae7b32175 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:48.739116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229d38-1526-43dc-b0c4-f52ae7b32175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:48.770061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-37229d38-1526-43dc-b0c4-f52ae7b32175 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:48.772911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-37229d38-1526-43dc-b0c4-f52ae7b32175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:48.990942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1067/2167] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:45:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:49.030530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:49.032828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:49.273657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db4802-3409-4c76-978d-3062cea18ffa service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:49.280167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:49.280909 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:49.281258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:49.281671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:49.291982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:49.294135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:49.328170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1101/2168] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:45:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:45:49.439913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244e22c-f8a3-4c7d-8c27-e5031b53a215 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:49.440322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244e22c-f8a3-4c7d-8c27-e5031b53a215 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:49.440698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244e22c-f8a3-4c7d-8c27-e5031b53a215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:49.492332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f244e22c-f8a3-4c7d-8c27-e5031b53a215 None None] MFA Rules not processed for user `2c917c21c7974c8d9746e8b05d9c8b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:49.494658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f244e22c-f8a3-4c7d-8c27-e5031b53a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:49.643160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1068/2169] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:50.245194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:50.275865 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:50.394686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352b82-f6ea-4a9e-b8ad-2190d4fa8d02 service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:50.400714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:50.401811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:50.402343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:50.402842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] PATH_INFO: `/v3/users/9acdb666b2a044bca60c01564fc050a5` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:50.405009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acdb666b2a044bca60c01564fc050a5)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:50.407535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:50.539163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acdb666b2a044bca60c01564fc050a5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:50.539163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acdb666b2a044bca60c01564fc050a5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:50.540518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acdb666b2a044bca60c01564fc050a5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:50.554843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acdb666b2a044bca60c01564fc050a5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:50.622083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acdb666b2a044bca60c01564fc050a5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:50.663804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1102/2170] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:45:50 2022] DELETE /identity/v3/users/9acdb666b2a044bca60c01564fc050a5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:50.669764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:50.670367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:50.670682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:50.748605 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:50.749919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:50.937993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1069/2171] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:50.952022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:50.953497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:51.100598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d29f99-b502-4c7b-bd85-ba9a92ea0f8a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:51.114443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:51.115010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:51.115930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:51.116357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:51.140942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:51.142629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:51.182541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1103/2172] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:51.815101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:51.822800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:51.857606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646afe7-c0e6-4a3b-a6e6-e92db91b591b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:51.869212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:51.872950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:51.873429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:51.873868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] PATH_INFO: `/v3/projects/7139329fd50d4910820437fe17d09a1a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:51.880185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7139329fd50d4910820437fe17d09a1a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:51.882681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:52.003648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:52.019769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7139329fd50d4910820437fe17d09a1a', 'request_id': 'req-a11b2c04-bf10-4263-8470-87a629acc1a6'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.032897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7139329fd50d4910820437fe17d09a1a', 'request_id': 'req-a11b2c04-bf10-4263-8470-87a629acc1a6'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.046887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7139329fd50d4910820437fe17d09a1a', 'request_id': 'req-a11b2c04-bf10-4263-8470-87a629acc1a6'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.096421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] The token cache is being invalidate because project 7139329fd50d4910820437fe17d09a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:45:52.096421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.099604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.102890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1070/2173] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:45:51 2022] DELETE /identity/v3/projects/7139329fd50d4910820437fe17d09a1a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:52.106865 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:52.121352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e64b-7fd3-4f2b-828a-8881319e5310 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:52.207454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:52.308918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:52.309624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:52.309918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:52.310201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] PATH_INFO: `/v3/users/4a2de621c3934549a432ebc68e3c2941` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:52.315975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2de621c3934549a432ebc68e3c2941)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:52.318012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:52.466338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2de621c3934549a432ebc68e3c2941'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.472043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2de621c3934549a432ebc68e3c2941'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.514328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2de621c3934549a432ebc68e3c2941'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.515258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2de621c3934549a432ebc68e3c2941'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.532405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5c877e71-d635-4577-9037-f6e6afbca1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2de621c3934549a432ebc68e3c2941'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:52.594685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1104/2174] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:45:52 2022] DELETE /identity/v3/users/4a2de621c3934549a432ebc68e3c2941 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:54.065592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:54.067889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:54.101806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b2c04-bf10-4263-8470-87a629acc1a6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:54.337754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:54.338797 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:54.339182 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:54.341866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] PATH_INFO: `/v3/projects/949bf53270bc46bdace845512886eadb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:54.344865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=949bf53270bc46bdace845512886eadb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:54.351023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:54.512841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '949bf53270bc46bdace845512886eadb', 'request_id': 'req-177ece64-8cfd-4e95-a755-960c9f21fbc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:54.536842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '949bf53270bc46bdace845512886eadb', 'request_id': 'req-177ece64-8cfd-4e95-a755-960c9f21fbc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:54.565446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '949bf53270bc46bdace845512886eadb', 'request_id': 'req-177ece64-8cfd-4e95-a755-960c9f21fbc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:54.624744 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] The token cache is being invalidate because project 949bf53270bc46bdace845512886eadb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:45:54.625015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:54.625658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-177ece64-8cfd-4e95-a755-960c9f21fbc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:54.636317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1071/2175] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:45:54 2022] DELETE /identity/v3/projects/949bf53270bc46bdace845512886eadb => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:54.638233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1bc70-cfc3-4307-8c0a-11949e8ca24b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:54.638758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1bc70-cfc3-4307-8c0a-11949e8ca24b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:54.638876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1bc70-cfc3-4307-8c0a-11949e8ca24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:54.695919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-39e1bc70-cfc3-4307-8c0a-11949e8ca24b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:54.697307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-39e1bc70-cfc3-4307-8c0a-11949e8ca24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:54.904777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1105/2176] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:54.913559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2edebd-97c7-4e5b-9abe-9fbdebbc4a6f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:54.913854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2edebd-97c7-4e5b-9abe-9fbdebbc4a6f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:54.914141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2edebd-97c7-4e5b-9abe-9fbdebbc4a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:54.957782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-cf2edebd-97c7-4e5b-9abe-9fbdebbc4a6f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:54.959070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cf2edebd-97c7-4e5b-9abe-9fbdebbc4a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:54.986265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1072/2177] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:54.991544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1bc70-cfc3-4307-8c0a-11949e8ca24b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:55.001929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1bc70-cfc3-4307-8c0a-11949e8ca24b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:55.035466 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:55.037387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:55.037387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:55.037387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.038130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:45:55.045974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:55.048091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:55.078796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1106/2178] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:45:55.080362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2edebd-97c7-4e5b-9abe-9fbdebbc4a6f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:55.098986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2edebd-97c7-4e5b-9abe-9fbdebbc4a6f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:55.105349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:55.106277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:55.106873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:55.107577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.108442 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:55.111233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:55.191098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:55.201931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1073/2179] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:45:55 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:55.205686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d054bd3-0f47-435b-b753-5652cef4161e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:55.334875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:55.416414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:55.417254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:55.419252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:55.419657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.420442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:55.422602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:55.551574 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1107/2180] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:45:55 2022] POST /identity/v3/users => generated 291 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:55.553753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:55.587986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9abe74-bc4a-43d1-8a0c-2cb1df1e1248 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:55.597344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:55.598176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:55.598582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:55.598946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.600133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:55.602722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:55.633285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1074/2181] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:55.637058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:55.653641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb4fa4-4b79-4881-b5da-0d2bc3b13121 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:55.669391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e315016-1527-48df-97ec-e29380988af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:55.670234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315016-1527-48df-97ec-e29380988af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:55.670623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315016-1527-48df-97ec-e29380988af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:55.671036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e315016-1527-48df-97ec-e29380988af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.671834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315016-1527-48df-97ec-e29380988af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:55.675647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e315016-1527-48df-97ec-e29380988af5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:55.709324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1108/2182] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:55.717239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:55.758244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9b606-b813-4da3-a28f-623553e8d8c9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:55.801122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:55.802085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:55.802825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:55.803362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] PATH_INFO: `/v3/projects/afd56ef2630a4bacaad299396d152d6a/users/8a52e1d37ea94a9898825af2cf974ae7/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.863116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd56ef2630a4bacaad299396d152d6a, user_id=8a52e1d37ea94a9898825af2cf974ae7, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:55.867167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:55.871576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:55.899489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1075/2183] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:55 2022] PUT /identity/v3/projects/afd56ef2630a4bacaad299396d152d6a/users/8a52e1d37ea94a9898825af2cf974ae7/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:55.916728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa047-90d1-48c0-9dde-bcdcbd0ea413 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:55.917521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa047-90d1-48c0-9dde-bcdcbd0ea413 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:55.917973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa047-90d1-48c0-9dde-bcdcbd0ea413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.980170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-86afa047-90d1-48c0-9dde-bcdcbd0ea413 None None] MFA Rules not processed for user `8a52e1d37ea94a9898825af2cf974ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:55.981783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-86afa047-90d1-48c0-9dde-bcdcbd0ea413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:56.143542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1109/2184] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:56.148593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:56.309496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319558e9-67b3-4e63-9b04-1bae45bb642b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:56.317737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:56.318471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:56.318788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:56.319027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:56.319711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:56.321871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:56.399257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1076/2185] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:45:56 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:56.403947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afa047-90d1-48c0-9dde-bcdcbd0ea413 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:56.423302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afa047-90d1-48c0-9dde-bcdcbd0ea413 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:56.535952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:56.536604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:56.536881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:56.537177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:56.537807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:56.550656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:56.639006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1110/2186] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:45:56 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:45:56.643445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:56.657489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e9f22-8b9d-4496-8002-fef9b20a452c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:56.662101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:56.662893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:56.663180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:56.663528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:56.664218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:56.666615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:56.685258 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1077/2187] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:56.687460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:56.700564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e604b7-e3af-44e8-9c50-facadb9a2cf8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:56.705872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:56.706791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:56.707405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:56.707855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] PATH_INFO: `/v3/projects/229892ed625e4189a4eb9994c9073960/users/5ee49297307f4fcc80dc1d41b61b1233/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:56.758643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=229892ed625e4189a4eb9994c9073960, user_id=5ee49297307f4fcc80dc1d41b61b1233, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:56.761676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:56.791420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1111/2188] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:56 2022] PUT /identity/v3/projects/229892ed625e4189a4eb9994c9073960/users/5ee49297307f4fcc80dc1d41b61b1233/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:56.793337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:56.811821 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6d276-6fb3-472b-a7c0-91d0f4f5a54e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:56.939532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb8894-669e-46a6-9f87-59d0206744ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:56.940169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8894-669e-46a6-9f87-59d0206744ab admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:56.940456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8894-669e-46a6-9f87-59d0206744ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:56.940735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb8894-669e-46a6-9f87-59d0206744ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:56.941631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb8894-669e-46a6-9f87-59d0206744ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:56.944243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb8894-669e-46a6-9f87-59d0206744ab admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:56.956782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1078/2189] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:45:56.962274 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:56.983248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81398f2a-e60f-4a66-84de-8ea20a3e16d7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:57.000136 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:57.000843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:57.001707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:57.002156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] PATH_INFO: `/v3/projects/229892ed625e4189a4eb9994c9073960/users/5ee49297307f4fcc80dc1d41b61b1233/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:57.008510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=229892ed625e4189a4eb9994c9073960, user_id=5ee49297307f4fcc80dc1d41b61b1233, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:57.013521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:57.037713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:57.054698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1112/2190] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:45:56 2022] PUT /identity/v3/projects/229892ed625e4189a4eb9994c9073960/users/5ee49297307f4fcc80dc1d41b61b1233/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:57.056909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2905f45-e208-4cac-975b-ac22fa8a3b07 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:57.059754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2905f45-e208-4cac-975b-ac22fa8a3b07 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:57.061825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2905f45-e208-4cac-975b-ac22fa8a3b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:57.139549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a2905f45-e208-4cac-975b-ac22fa8a3b07 None None] MFA Rules not processed for user `5ee49297307f4fcc80dc1d41b61b1233`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:57.142055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a2905f45-e208-4cac-975b-ac22fa8a3b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:57.287166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:57.339117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1079/2191] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:57.359509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:57.361404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:57.497085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb07ea6-28c8-4f18-9681-8ab172fb56aa admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:57.504370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c421b48-b0be-410a-8f6b-1c2d15ee208b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:57.505050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c421b48-b0be-410a-8f6b-1c2d15ee208b service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:57.505345 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c421b48-b0be-410a-8f6b-1c2d15ee208b service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:57.505604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c421b48-b0be-410a-8f6b-1c2d15ee208b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:57.528863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c421b48-b0be-410a-8f6b-1c2d15ee208b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:57.530605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c421b48-b0be-410a-8f6b-1c2d15ee208b service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:57.545822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1113/2192] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:58.629507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2905f45-e208-4cac-975b-ac22fa8a3b07 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:58.818738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2905f45-e208-4cac-975b-ac22fa8a3b07 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:58.825240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:58.826114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:58.826412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:58.826697 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] PATH_INFO: `/v3/users/8a52e1d37ea94a9898825af2cf974ae7` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:58.827923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a52e1d37ea94a9898825af2cf974ae7)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:58.831092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:58.966089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a52e1d37ea94a9898825af2cf974ae7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:58.986984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a52e1d37ea94a9898825af2cf974ae7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:59.019813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a52e1d37ea94a9898825af2cf974ae7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:59.020446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a52e1d37ea94a9898825af2cf974ae7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:59.023788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a52e1d37ea94a9898825af2cf974ae7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:45:59.068269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1080/2193] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:45:58 2022] DELETE /identity/v3/users/8a52e1d37ea94a9898825af2cf974ae7 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:45:59.074587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:59.074587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:59.074587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:59.106517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:59.107719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:59.320726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1114/2194] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:59.321550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:59.323759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:59.465919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d144d28-8eea-40e1-a2a9-a650ed56a85f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:59.470520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:59.471631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:59.472160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:59.472663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:59.495666 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:59.498396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:59.525590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1081/2195] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:00.348217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:00.349840 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:00.381581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7d8af-42ab-43e0-bfa0-fc451a0cc9a4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:00.394639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:00.395763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:00.396226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:00.396686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] PATH_INFO: `/v3/projects/afd56ef2630a4bacaad299396d152d6a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:00.403129 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd56ef2630a4bacaad299396d152d6a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:00.405725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:00.540842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd56ef2630a4bacaad299396d152d6a', 'request_id': 'req-deffd655-73b7-4bb4-89ce-8b51af0e73b9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.558864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd56ef2630a4bacaad299396d152d6a', 'request_id': 'req-deffd655-73b7-4bb4-89ce-8b51af0e73b9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.571041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd56ef2630a4bacaad299396d152d6a', 'request_id': 'req-deffd655-73b7-4bb4-89ce-8b51af0e73b9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.596608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] The token cache is being invalidate because project afd56ef2630a4bacaad299396d152d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:00.597043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.598060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.601699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1115/2196] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:00 2022] DELETE /identity/v3/projects/afd56ef2630a4bacaad299396d152d6a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:00.609488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:00.633151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c265e-7c81-44b5-b45c-2793cb78a933 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:00.681980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:00.764104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:00.764795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:00.765085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:00.765370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] PATH_INFO: `/v3/users/5ee49297307f4fcc80dc1d41b61b1233` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:00.766566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee49297307f4fcc80dc1d41b61b1233)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:00.768864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:00.858091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee49297307f4fcc80dc1d41b61b1233'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.871404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee49297307f4fcc80dc1d41b61b1233'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.905431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee49297307f4fcc80dc1d41b61b1233'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.910387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee49297307f4fcc80dc1d41b61b1233'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:00.911643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0e9b568-4b4a-4921-8a41-c570fdd9726f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee49297307f4fcc80dc1d41b61b1233'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:01.045189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1082/2197] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:00 2022] DELETE /identity/v3/users/5ee49297307f4fcc80dc1d41b61b1233 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:01.861438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:01.863892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:01.885208 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffd655-73b7-4bb4-89ce-8b51af0e73b9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:01.943964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:02.018563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:02.019243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:02.019532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:02.019813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] PATH_INFO: `/v3/projects/229892ed625e4189a4eb9994c9073960` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:02.022024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] RBAC: Authorizing `identity:delete_project(project_id=229892ed625e4189a4eb9994c9073960)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:02.024158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:02.128678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229892ed625e4189a4eb9994c9073960', 'request_id': 'req-bbebb9df-f21c-4c88-8701-1436e31be63c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:02.140711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229892ed625e4189a4eb9994c9073960', 'request_id': 'req-bbebb9df-f21c-4c88-8701-1436e31be63c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:02.158376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229892ed625e4189a4eb9994c9073960', 'request_id': 'req-bbebb9df-f21c-4c88-8701-1436e31be63c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:02.178659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] The token cache is being invalidate because project 229892ed625e4189a4eb9994c9073960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:02.178959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:02.182541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bbebb9df-f21c-4c88-8701-1436e31be63c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:02.189449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1116/2198] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:01 2022] DELETE /identity/v3/projects/229892ed625e4189a4eb9994c9073960 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:02.196564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc35b3-f9b4-4fdd-bb0d-b7ca857bfe46 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:02.197300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc35b3-f9b4-4fdd-bb0d-b7ca857bfe46 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:02.197399 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc35b3-f9b4-4fdd-bb0d-b7ca857bfe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:02.232529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bdcc35b3-f9b4-4fdd-bb0d-b7ca857bfe46 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:02.233858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bdcc35b3-f9b4-4fdd-bb0d-b7ca857bfe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:02.350999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1083/2199] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:02.353209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b720-e620-4ff8-aa3a-65c62e357478 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:02.353485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b720-e620-4ff8-aa3a-65c62e357478 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:02.353767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b720-e620-4ff8-aa3a-65c62e357478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:02.385300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-26d6b720-e620-4ff8-aa3a-65c62e357478 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:02.393267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-26d6b720-e620-4ff8-aa3a-65c62e357478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:02.412317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1117/2200] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:02.416959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc35b3-f9b4-4fdd-bb0d-b7ca857bfe46 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:02.433367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc35b3-f9b4-4fdd-bb0d-b7ca857bfe46 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:02.459950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:02.460906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:02.461389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:02.461857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:02.464321 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:46:02.466331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:02.468300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:02.494915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1084/2201] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:46:02.499401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6b720-e620-4ff8-aa3a-65c62e357478 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:02.520368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6b720-e620-4ff8-aa3a-65c62e357478 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:02.535938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:02.536593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:02.539337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:02.539647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:02.540356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:02.542323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:02.606967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1118/2202] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:02 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:02.611420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:02.626554 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c40ca-a2f3-442b-bcad-02af53330fc0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:02.664856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:02.730692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:02.731551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:02.731824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:02.734177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:02.734177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:02.735422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:02.812209 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1085/2203] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:02 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:02.812906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:02.863815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e689c58-2164-4c43-9e41-e7923493b9d6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:02.910303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:02.910936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:02.911235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:02.911515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:02.912184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:02.913831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:02.953445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1119/2204] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:02.955682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:03.032424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b2f34-9b9b-4614-9990-d4c73d586675 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:03.048142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d393c-6270-4a3a-a9f0-59e5f607eb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:03.049097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d393c-6270-4a3a-a9f0-59e5f607eb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:03.049536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d393c-6270-4a3a-a9f0-59e5f607eb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:03.049968 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d393c-6270-4a3a-a9f0-59e5f607eb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:03.050804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d393c-6270-4a3a-a9f0-59e5f607eb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:03.052702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d393c-6270-4a3a-a9f0-59e5f607eb8f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:03.093460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1086/2205] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:03.098147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:03.129668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96954d4-5b81-43e5-9109-40bab1b5dad3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:03.139545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:03.141327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:03.141787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:03.142232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] PATH_INFO: `/v3/projects/741f0c811f144ef0bad7a7c8b9fa380e/users/ad1eab7473404021a0142b6eca7f599f/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:03.166842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741f0c811f144ef0bad7a7c8b9fa380e, user_id=ad1eab7473404021a0142b6eca7f599f, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:03.169466 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:03.198167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1120/2206] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:03 2022] PUT /identity/v3/projects/741f0c811f144ef0bad7a7c8b9fa380e/users/ad1eab7473404021a0142b6eca7f599f/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:03.212047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d2eb-5d9f-47ac-8666-cdd6bc67e9fa None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:03.212944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d2eb-5d9f-47ac-8666-cdd6bc67e9fa None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:03.232692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d2eb-5d9f-47ac-8666-cdd6bc67e9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:03.272688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d4e3d2eb-5d9f-47ac-8666-cdd6bc67e9fa None None] MFA Rules not processed for user `ad1eab7473404021a0142b6eca7f599f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:03.273932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d4e3d2eb-5d9f-47ac-8666-cdd6bc67e9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:03.438502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1087/2207] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:03.442573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:03.579504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:03.604094 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef392ee4-11df-43cf-8c62-115791633224 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:03.620577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:03.624432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:03.624726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:03.625154 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:03.625905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:03.628256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:03.706167 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1121/2208] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:03 2022] POST /identity/v3/projects => generated 373 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:03.710436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3d2eb-5d9f-47ac-8666-cdd6bc67e9fa None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:03.735342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3d2eb-5d9f-47ac-8666-cdd6bc67e9fa None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:03.917503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:03.918322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:03.918712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:03.919092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:03.921497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:03.925220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:03.981040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1088/2209] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:03 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:03.984677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:04.006200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074716e-cca7-434a-9a6b-9378ddeddea4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:04.030713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:04.031390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:04.031668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:04.031941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:04.032612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:04.034318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:04.059339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1122/2210] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:04.060411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:04.074671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8df89-b9bd-49f8-8c05-db7c6903c228 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:04.090288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:04.090982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:04.091306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:04.091590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] PATH_INFO: `/v3/projects/8e2a1b2ad04341898a8ef400eccd9b07/users/e3ea4025ef734ac4ae37ce6425656ce3/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:04.141179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2a1b2ad04341898a8ef400eccd9b07, user_id=e3ea4025ef734ac4ae37ce6425656ce3, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:04.146173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:04.149024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:04.185678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:04.190495 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1089/2211] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:04 2022] PUT /identity/v3/projects/8e2a1b2ad04341898a8ef400eccd9b07/users/e3ea4025ef734ac4ae37ce6425656ce3/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:46:04.242705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e75e5-9474-40b7-b251-268004b809a6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:04.379452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46d2d-6ba9-4511-92f4-f28dd52d358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:04.381652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46d2d-6ba9-4511-92f4-f28dd52d358d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:04.382032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46d2d-6ba9-4511-92f4-f28dd52d358d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:04.382381 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46d2d-6ba9-4511-92f4-f28dd52d358d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:04.385437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f46d2d-6ba9-4511-92f4-f28dd52d358d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:04.388509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f46d2d-6ba9-4511-92f4-f28dd52d358d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:04.406457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1123/2212] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:04.407023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:04.418206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6157-609e-48a5-9821-e52a2bd558f9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:04.439739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:04.440443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:04.440729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:04.441012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] PATH_INFO: `/v3/projects/8e2a1b2ad04341898a8ef400eccd9b07/users/e3ea4025ef734ac4ae37ce6425656ce3/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:04.443018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2a1b2ad04341898a8ef400eccd9b07, user_id=e3ea4025ef734ac4ae37ce6425656ce3, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:04.446046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:04.479237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1090/2213] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:04 2022] PUT /identity/v3/projects/8e2a1b2ad04341898a8ef400eccd9b07/users/e3ea4025ef734ac4ae37ce6425656ce3/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:04.486311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774da32-afe2-4952-9e10-4b190987d7d1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:04.486671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774da32-afe2-4952-9e10-4b190987d7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:04.486977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774da32-afe2-4952-9e10-4b190987d7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:04.561704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c774da32-afe2-4952-9e10-4b190987d7d1 None None] MFA Rules not processed for user `e3ea4025ef734ac4ae37ce6425656ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:04.563025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c774da32-afe2-4952-9e10-4b190987d7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:04.728301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1124/2214] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:04.752446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:04.759655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:04.835614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b525b1f-4bc9-46b5-b2aa-e15157b89ff2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:04.840251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838405df-5b58-4b4d-a0be-a0448fe19d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:04.841033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838405df-5b58-4b4d-a0be-a0448fe19d30 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:04.841414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838405df-5b58-4b4d-a0be-a0448fe19d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:04.841794 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838405df-5b58-4b4d-a0be-a0448fe19d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:04.932586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838405df-5b58-4b4d-a0be-a0448fe19d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:04.932586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838405df-5b58-4b4d-a0be-a0448fe19d30 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:04.941200 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1091/2215] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:06.496173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:06.496671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:06.496769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:06.528473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] MFA Rules not processed for user `686bbf803837400e94f1df1efd04a407`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:06.531474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:06.620497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:06.639827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1125/2216] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:06.926879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8068a2-29ab-494d-a1e5-4118053b37e7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:06.927490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8068a2-29ab-494d-a1e5-4118053b37e7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:06.927490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8068a2-29ab-494d-a1e5-4118053b37e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:06.955115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dc8068a2-29ab-494d-a1e5-4118053b37e7 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:06.956459 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dc8068a2-29ab-494d-a1e5-4118053b37e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:07.019164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1092/2217] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 06:46:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:07.033435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:07.036057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:07.115757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748b339-97eb-4e35-bbe9-6ea93409f64d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:07.122522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ea8e6-50a6-4210-9c5e-448c2e9c0c5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:07.123439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea8e6-50a6-4210-9c5e-448c2e9c0c5e service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:07.123871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea8e6-50a6-4210-9c5e-448c2e9c0c5e service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:07.124309 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ea8e6-50a6-4210-9c5e-448c2e9c0c5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:07.135623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea8e6-50a6-4210-9c5e-448c2e9c0c5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:07.138091 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ea8e6-50a6-4210-9c5e-448c2e9c0c5e service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:07.148371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1126/2218] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:46:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:46:07.716418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8068a2-29ab-494d-a1e5-4118053b37e7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:07.863361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:07.917159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8068a2-29ab-494d-a1e5-4118053b37e7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:07.932445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:07.934103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:07.934708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:07.935180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] PATH_INFO: `/v3/users/ad1eab7473404021a0142b6eca7f599f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:07.938192 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1eab7473404021a0142b6eca7f599f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:07.941788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:08.037038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1eab7473404021a0142b6eca7f599f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:08.062152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1eab7473404021a0142b6eca7f599f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:08.086046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1eab7473404021a0142b6eca7f599f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:08.088864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1eab7473404021a0142b6eca7f599f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:08.090296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1eab7473404021a0142b6eca7f599f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:08.128557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1093/2219] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:07 2022] DELETE /identity/v3/users/ad1eab7473404021a0142b6eca7f599f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:08.132905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:08.133398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:08.133614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:08.155128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:08.156939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:08.329183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1127/2220] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:08.340970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:08.342775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:08.435033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94267ef3-7bdb-4c9e-afa0-d17f60c8e3fc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:08.440346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:08.441403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:08.441916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:08.442417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:08.468342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:08.472349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:08.501741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1094/2221] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:09.950628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:09.962974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:09.986295 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8596c-4071-45ad-8499-d73ca3cafad4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:09.993515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:09.994965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:09.995509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:09.995936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] PATH_INFO: `/v3/projects/741f0c811f144ef0bad7a7c8b9fa380e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:09.997509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741f0c811f144ef0bad7a7c8b9fa380e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:10.000420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:10.106637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741f0c811f144ef0bad7a7c8b9fa380e', 'request_id': 'req-87ee1da2-7f4f-4e38-880f-b500190e77a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.126683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741f0c811f144ef0bad7a7c8b9fa380e', 'request_id': 'req-87ee1da2-7f4f-4e38-880f-b500190e77a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.137016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741f0c811f144ef0bad7a7c8b9fa380e', 'request_id': 'req-87ee1da2-7f4f-4e38-880f-b500190e77a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.163823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] The token cache is being invalidate because project 741f0c811f144ef0bad7a7c8b9fa380e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:10.164696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.165815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.173017 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1128/2222] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:09 2022] DELETE /identity/v3/projects/741f0c811f144ef0bad7a7c8b9fa380e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:10.174060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:10.187698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56bc68-9ede-4c6a-8cbe-ba2ea1fc9066 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:10.280562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:10.281536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:10.281967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:10.282428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] PATH_INFO: `/v3/users/e3ea4025ef734ac4ae37ce6425656ce3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:10.284123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ea4025ef734ac4ae37ce6425656ce3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:10.286562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:10.289933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:10.373985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ea4025ef734ac4ae37ce6425656ce3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.404313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ea4025ef734ac4ae37ce6425656ce3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.420894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ea4025ef734ac4ae37ce6425656ce3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.422009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ea4025ef734ac4ae37ce6425656ce3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.422983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ea4025ef734ac4ae37ce6425656ce3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.486211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1095/2223] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:10 2022] DELETE /identity/v3/users/e3ea4025ef734ac4ae37ce6425656ce3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:10.556197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:10.557891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:10.589318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee1da2-7f4f-4e38-880f-b500190e77a3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:10.600434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:10.692249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:10.693338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:10.693804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:10.694917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] PATH_INFO: `/v3/users/814c0227c728493ebd4364bc154cee83` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:10.696728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=814c0227c728493ebd4364bc154cee83)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:10.699079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:10.774786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814c0227c728493ebd4364bc154cee83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.777168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814c0227c728493ebd4364bc154cee83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.786780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814c0227c728493ebd4364bc154cee83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.787982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814c0227c728493ebd4364bc154cee83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.799372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814c0227c728493ebd4364bc154cee83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:10.823973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1129/2224] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:10 2022] DELETE /identity/v3/users/814c0227c728493ebd4364bc154cee83 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:11.118385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:11.120233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:11.151042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29ea5e-271a-4080-bc91-e97867ec68e1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:11.364987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:11.365694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:11.365933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:11.369836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] PATH_INFO: `/v3/projects/0930b613a69749d3bc05240e88f74ccc` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:11.374238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0930b613a69749d3bc05240e88f74ccc)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:11.385080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:11.500994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0930b613a69749d3bc05240e88f74ccc', 'request_id': 'req-2dcece08-f02f-4056-90f8-00ddd7399833'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:11.515525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0930b613a69749d3bc05240e88f74ccc', 'request_id': 'req-2dcece08-f02f-4056-90f8-00ddd7399833'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:11.526435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0930b613a69749d3bc05240e88f74ccc', 'request_id': 'req-2dcece08-f02f-4056-90f8-00ddd7399833'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:11.561342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] The token cache is being invalidate because project 0930b613a69749d3bc05240e88f74ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:11.561817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:11.563501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2dcece08-f02f-4056-90f8-00ddd7399833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:11.565385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1096/2225] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:11 2022] DELETE /identity/v3/projects/0930b613a69749d3bc05240e88f74ccc => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:11.882074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:11.883604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:11.919503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde739bc-0636-4809-8d8b-8f75fffe61bf admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:12.036930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:12.037269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.037653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.038080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] PATH_INFO: `/v3/projects/8e2a1b2ad04341898a8ef400eccd9b07` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.055168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2a1b2ad04341898a8ef400eccd9b07)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:12.075638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:12.144267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:12.176525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2a1b2ad04341898a8ef400eccd9b07', 'request_id': 'req-eec68ac7-1847-4116-9846-3662e55ccda0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:12.189033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2a1b2ad04341898a8ef400eccd9b07', 'request_id': 'req-eec68ac7-1847-4116-9846-3662e55ccda0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:12.200522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2a1b2ad04341898a8ef400eccd9b07', 'request_id': 'req-eec68ac7-1847-4116-9846-3662e55ccda0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:12.227258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] The token cache is being invalidate because project 8e2a1b2ad04341898a8ef400eccd9b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:12.227897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:12.231285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-eec68ac7-1847-4116-9846-3662e55ccda0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:12.239100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1130/2226] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:11 2022] DELETE /identity/v3/projects/8e2a1b2ad04341898a8ef400eccd9b07 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:12.241008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a922a-8b3d-4477-9feb-c8536d3d524a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.241286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a922a-8b3d-4477-9feb-c8536d3d524a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.241581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a922a-8b3d-4477-9feb-c8536d3d524a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.288885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f14a922a-8b3d-4477-9feb-c8536d3d524a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:12.290397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f14a922a-8b3d-4477-9feb-c8536d3d524a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:12.307490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:12.392434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1097/2227] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:12.400374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a611-7cbd-4a3a-9c45-6ed496135f13 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.400784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a611-7cbd-4a3a-9c45-6ed496135f13 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.401070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a611-7cbd-4a3a-9c45-6ed496135f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.424376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0d18a611-7cbd-4a3a-9c45-6ed496135f13 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:12.428580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0d18a611-7cbd-4a3a-9c45-6ed496135f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:12.438959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1131/2228] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:12.443223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a922a-8b3d-4477-9feb-c8536d3d524a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:12.454413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a922a-8b3d-4477-9feb-c8536d3d524a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:12.459749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:12.460521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.460865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.461205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.462074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:46:12.462454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:12.464498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:12.486314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1098/2229] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:46:12.487559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a611-7cbd-4a3a-9c45-6ed496135f13 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:12.500404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a611-7cbd-4a3a-9c45-6ed496135f13 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:12.506299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:12.508100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.509053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.509053 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.509728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:12.512736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:12.570442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1132/2230] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:12 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:12.573999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:12.593882 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359002a7-e5a6-4f8a-9d27-1963107546a2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:12.703850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:12.704751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.704933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.705327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.707017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:12.711754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:12.764426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1099/2231] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:12 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:12.769490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:12.789163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60353b6f-805e-4bf5-8689-2cd24431bb90 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:12.799044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:12.802256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.802861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.803606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.804473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:12.807567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:12.826943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1133/2232] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:12.827648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:12.849388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088d41a-67e1-4bb2-a09c-cc729c9970b7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:12.856453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8abf32-0392-4384-baa6-d73d34f903ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:12.857310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8abf32-0392-4384-baa6-d73d34f903ab admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.858550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8abf32-0392-4384-baa6-d73d34f903ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.859030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8abf32-0392-4384-baa6-d73d34f903ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.859921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8abf32-0392-4384-baa6-d73d34f903ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:12.862133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8abf32-0392-4384-baa6-d73d34f903ab admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:12.885713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:12.896756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1100/2233] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:12.906582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837d1ff-daca-4281-9d32-81b5d736d5b5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:12.912474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:12.913399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.913870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.914602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] PATH_INFO: `/v3/projects/831ccf2d1e994dd1ab0e7e88eff00af8/users/98196cc45c404c028494189852d9c5f1/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:12.944435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=831ccf2d1e994dd1ab0e7e88eff00af8, user_id=98196cc45c404c028494189852d9c5f1, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:12.952733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:12.970971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1134/2234] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:12 2022] PUT /identity/v3/projects/831ccf2d1e994dd1ab0e7e88eff00af8/users/98196cc45c404c028494189852d9c5f1/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:12.982451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab25bf-8cbf-4a0c-8843-af6423f9abdd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:12.982746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab25bf-8cbf-4a0c-8843-af6423f9abdd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:12.983015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab25bf-8cbf-4a0c-8843-af6423f9abdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.015687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-62ab25bf-8cbf-4a0c-8843-af6423f9abdd None None] MFA Rules not processed for user `98196cc45c404c028494189852d9c5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:13.016924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-62ab25bf-8cbf-4a0c-8843-af6423f9abdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:13.108875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1101/2235] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:13.113611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:13.238993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50744f20-8a87-4dff-aaf5-c318fd33c4c8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:13.251599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:13.252338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.252791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.253140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.253890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:13.257726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:13.335782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab25bf-8cbf-4a0c-8843-af6423f9abdd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:13.346070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1135/2236] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:13 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:13.350549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab25bf-8cbf-4a0c-8843-af6423f9abdd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:13.355839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:13.425560 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:13.426281 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.427097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.427097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.427735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:13.430199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:13.460798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1102/2237] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:13 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:13.462581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:13.480098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e128-7f15-4c05-97c2-3163ba7de78b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:13.493450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:13.494699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.494987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.495291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.495960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:13.503388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:13.517085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1136/2238] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:13.521509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:13.534380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694eb26-75e4-4cdc-bd5a-5a885bc7a0c1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:13.542335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bd23d-25c8-465c-8621-7b9aa74059d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:13.544649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bd23d-25c8-465c-8621-7b9aa74059d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.545724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bd23d-25c8-465c-8621-7b9aa74059d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.546109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bd23d-25c8-465c-8621-7b9aa74059d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.546716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bd23d-25c8-465c-8621-7b9aa74059d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:13.548554 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bd23d-25c8-465c-8621-7b9aa74059d5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:13.573612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1103/2239] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:13.574688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:13.585899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9bde0-24c2-43c0-a679-b484d5dc1efe admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:13.591526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:13.592200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.592484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.592764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] PATH_INFO: `/v3/projects/c7fe8b98bf5843319f6cf9bd941ec642/users/07c625af41044628ac45eed1e9ce9a60/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.609324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fe8b98bf5843319f6cf9bd941ec642, user_id=07c625af41044628ac45eed1e9ce9a60, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:13.612077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:13.627302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1137/2240] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:13 2022] PUT /identity/v3/projects/c7fe8b98bf5843319f6cf9bd941ec642/users/07c625af41044628ac45eed1e9ce9a60/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:46:13.703581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271caa-6399-4ab5-a64c-4049fe5d5665 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.704041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271caa-6399-4ab5-a64c-4049fe5d5665 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.704206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271caa-6399-4ab5-a64c-4049fe5d5665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.735806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d5271caa-6399-4ab5-a64c-4049fe5d5665 None None] MFA Rules not processed for user `07c625af41044628ac45eed1e9ce9a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:13.737002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d5271caa-6399-4ab5-a64c-4049fe5d5665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:13.819884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1104/2241] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:13.824169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:13.830139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:13.932215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af69b6-49f0-4080-82ea-056601c8a7f2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:13.941503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272e680-b4cc-4ffc-b505-4a0f54cd19e9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.942061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272e680-b4cc-4ffc-b505-4a0f54cd19e9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.942507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272e680-b4cc-4ffc-b505-4a0f54cd19e9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.944127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1105/2242] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:46:13.952564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:13.953424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.953769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.953877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:13.954010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.954150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:13.954521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:13.955096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:13.960535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.008470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1138/2243] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:13 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:14.012978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.018187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:14.019543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:14.099716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2249c-dc09-4bde-9df7-dd1399400438 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.108571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.109513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.109901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.110327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.111662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.114998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.117978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1106/2244] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:46:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:14.131354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.133374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:14.146407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:14.155011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1139/2245] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:14 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:14.156751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.172173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c328f-7a63-43ed-860b-ebbe769f440b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.178704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.179503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.179795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.180064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.181301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.183268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.202181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1140/2246] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:14.207855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.228801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ab607-e01e-4baa-a1c7-be010ea7b631 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.235609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.236947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.237625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.238113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] PATH_INFO: `/v3/projects/5bed5f69c30946ee9fc4bb7a0af4f717/users/bccf4e8d12254bd7938ac622006c196f/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.257291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bed5f69c30946ee9fc4bb7a0af4f717, user_id=bccf4e8d12254bd7938ac622006c196f, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.261805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.287448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535c329-31fe-406e-a47c-07373e4d0edc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.297121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1141/2247] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:14 2022] PUT /identity/v3/projects/5bed5f69c30946ee9fc4bb7a0af4f717/users/bccf4e8d12254bd7938ac622006c196f/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:14.299146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.395797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:14.415350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6dfead-447d-4c89-a76b-37184652c0ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380786eb9b7848d59f3bb0007726dd2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.417548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dfead-447d-4c89-a76b-37184652c0ea service glance] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.418105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dfead-447d-4c89-a76b-37184652c0ea service glance] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.418846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dfead-447d-4c89-a76b-37184652c0ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.421314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3830e-b851-49c1-90b9-8e2075526820 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.426911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.427719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.428008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.428292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.428964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.431066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.443595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1142/2248] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:14.448080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.459210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368eb3a9-7aae-4a6e-acae-ef63c4afd5b9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.464505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1f914-17bb-4fb7-867b-9f8928c3be39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.465198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1f914-17bb-4fb7-867b-9f8928c3be39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.465475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1f914-17bb-4fb7-867b-9f8928c3be39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.465792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1f914-17bb-4fb7-867b-9f8928c3be39 admin admin] PATH_INFO: `/v3/projects/5bed5f69c30946ee9fc4bb7a0af4f717/users/bccf4e8d12254bd7938ac622006c196f/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.467757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1f914-17bb-4fb7-867b-9f8928c3be39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bed5f69c30946ee9fc4bb7a0af4f717, user_id=bccf4e8d12254bd7938ac622006c196f, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.470653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1f914-17bb-4fb7-867b-9f8928c3be39 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.487231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1143/2249] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:14 2022] PUT /identity/v3/projects/5bed5f69c30946ee9fc4bb7a0af4f717/users/bccf4e8d12254bd7938ac622006c196f/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:14.494740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.495184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.495504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.543421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] MFA Rules not processed for user `bccf4e8d12254bd7938ac622006c196f`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:14.546344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:14.547793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6dfead-447d-4c89-a76b-37184652c0ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.549571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6dfead-447d-4c89-a76b-37184652c0ea service glance] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.564394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1107/2250] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:46:14.678180 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1144/2251] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:14.683539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec4f8e-dbd9-4eaa-92c2-a38d124aa121 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.687377 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec4f8e-dbd9-4eaa-92c2-a38d124aa121 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.687873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec4f8e-dbd9-4eaa-92c2-a38d124aa121 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.692420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1108/2252] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:46:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:46:14.699023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.701786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:14.707546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.707908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.708197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.734941 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] MFA Rules not processed for user `2fc4592ef54b41059fa7767a5ca3aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:14.736476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:14.800133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:14.828262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c1d0c-0b48-4dcd-a712-3e4d46cb73c9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.833975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddfad7-2468-4bb3-8f81-44eec2adb2ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.834663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddfad7-2468-4bb3-8f81-44eec2adb2ee service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.834953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddfad7-2468-4bb3-8f81-44eec2adb2ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.835256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddfad7-2468-4bb3-8f81-44eec2adb2ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.839565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1109/2253] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:46:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:14.852086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:14.854983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:14.876053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5271db-10f1-411e-ad87-562bf1d6e508 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:14.881345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae0cce-682e-4c7e-b751-4ddec0ce4013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:14.881345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0cce-682e-4c7e-b751-4ddec0ce4013 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:14.881571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0cce-682e-4c7e-b751-4ddec0ce4013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:14.882037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0cce-682e-4c7e-b751-4ddec0ce4013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:14.897785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae0cce-682e-4c7e-b751-4ddec0ce4013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.902455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae0cce-682e-4c7e-b751-4ddec0ce4013 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.915910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1110/2254] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:46:14.917956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ddfad7-2468-4bb3-8f81-44eec2adb2ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:14.920184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ddfad7-2468-4bb3-8f81-44eec2adb2ee service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:14.932566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1145/2255] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:15.023588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e6b9-cfc3-40d9-8aa0-1b14a722d755 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:15.023879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e6b9-cfc3-40d9-8aa0-1b14a722d755 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:15.024161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e6b9-cfc3-40d9-8aa0-1b14a722d755 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:15.026717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1111/2256] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:46:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:46:15.033012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc738df-4f30-4b53-9d47-b70dab371e3c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:15.033305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc738df-4f30-4b53-9d47-b70dab371e3c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:15.033583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc738df-4f30-4b53-9d47-b70dab371e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:15.044408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:15.058537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-adc738df-4f30-4b53-9d47-b70dab371e3c None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:15.059828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-adc738df-4f30-4b53-9d47-b70dab371e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:15.163880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1146/2257] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:46:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:15.177775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864e6b9-cfc3-40d9-8aa0-1b14a722d755 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:15.195088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864e6b9-cfc3-40d9-8aa0-1b14a722d755 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:15.201360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:15.202140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:15.202478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:15.202715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:15.216419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:15.218327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:15.247643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1112/2258] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:15.757832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc738df-4f30-4b53-9d47-b70dab371e3c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:15.776418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc738df-4f30-4b53-9d47-b70dab371e3c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:15.780222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:15.781501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:15.781789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:15.782073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:15.784550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:15.862366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:15.863795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:15.874428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1147/2259] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:16.632248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:16.633958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:16.647644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5836e-fa1e-4199-a7e4-71ce94232761 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:16.653611 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13bb20-b1a4-4148-b60d-8ff4898125d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:16.654310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13bb20-b1a4-4148-b60d-8ff4898125d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:16.654635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13bb20-b1a4-4148-b60d-8ff4898125d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:16.655044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13bb20-b1a4-4148-b60d-8ff4898125d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:16.724534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13bb20-b1a4-4148-b60d-8ff4898125d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:16.726168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13bb20-b1a4-4148-b60d-8ff4898125d7 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:16.739029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1113/2260] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:17.430487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:17.441852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26811fce-f3bc-4d8b-9b35-daba815af902 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:17.445485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:17.446102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:17.446420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:17.446688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:17.455850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:17.457283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:17.467491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1148/2261] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:19.847508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63848f02-e653-47b3-9cd9-8b1ac266932c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:19.848001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63848f02-e653-47b3-9cd9-8b1ac266932c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:19.848094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63848f02-e653-47b3-9cd9-8b1ac266932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:19.867412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-63848f02-e653-47b3-9cd9-8b1ac266932c None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:19.868681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-63848f02-e653-47b3-9cd9-8b1ac266932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:19.927533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1114/2262] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:46:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:19.940837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:19.942293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:20.027960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d114b4-01e8-4553-b448-5f92a458903b service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:20.034034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfecae38-7ba3-449b-a6b9-610f9c58e38e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:20.034764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecae38-7ba3-449b-a6b9-610f9c58e38e service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:20.035139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecae38-7ba3-449b-a6b9-610f9c58e38e service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:20.035518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfecae38-7ba3-449b-a6b9-610f9c58e38e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:20.045828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecae38-7ba3-449b-a6b9-610f9c58e38e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:20.047546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfecae38-7ba3-449b-a6b9-610f9c58e38e service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:20.057374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1149/2263] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:46:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:46:21.592070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63848f02-e653-47b3-9cd9-8b1ac266932c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:21.675533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63848f02-e653-47b3-9cd9-8b1ac266932c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:21.680564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:21.681415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:21.681701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:21.681981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] PATH_INFO: `/v3/users/98196cc45c404c028494189852d9c5f1` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:21.683233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98196cc45c404c028494189852d9c5f1)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:21.685361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:21.700904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:21.761994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98196cc45c404c028494189852d9c5f1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:21.771006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98196cc45c404c028494189852d9c5f1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:21.785652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98196cc45c404c028494189852d9c5f1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:21.786434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98196cc45c404c028494189852d9c5f1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:21.787096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98196cc45c404c028494189852d9c5f1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:21.818199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1115/2264] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:21 2022] DELETE /identity/v3/users/98196cc45c404c028494189852d9c5f1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:21.820361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:21.822368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:21.822849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:21.849416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:21.860090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:21.861924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:21.962753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1150/2265] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:46:21.970621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:21.972249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:22.107065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca0546-6d6e-4e53-8cc0-65236c597423 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:22.110601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:22.112925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:22.113214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:22.113539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:22.132586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:22.135333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:22.146613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1116/2266] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:22.452712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:22.454232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:22.471614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb4d1e-25e1-4292-89d1-399a0c0b0a29 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:22.476674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:22.477336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:22.477856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:22.489330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] PATH_INFO: `/v3/projects/831ccf2d1e994dd1ab0e7e88eff00af8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:22.490826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=831ccf2d1e994dd1ab0e7e88eff00af8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:22.492966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:22.547649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '831ccf2d1e994dd1ab0e7e88eff00af8', 'request_id': 'req-5767a59e-4ead-4035-b070-37a03cca7df2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.554507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '831ccf2d1e994dd1ab0e7e88eff00af8', 'request_id': 'req-5767a59e-4ead-4035-b070-37a03cca7df2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.564150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '831ccf2d1e994dd1ab0e7e88eff00af8', 'request_id': 'req-5767a59e-4ead-4035-b070-37a03cca7df2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.588487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] The token cache is being invalidate because project 831ccf2d1e994dd1ab0e7e88eff00af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:22.588789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.589473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.591078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1151/2267] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:22 2022] DELETE /identity/v3/projects/831ccf2d1e994dd1ab0e7e88eff00af8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:22.595250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:22.604930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6869-f75b-4a92-9bce-5f7d7b1865d8 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:22.673838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:22.674452 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:22.674733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:22.675006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] PATH_INFO: `/v3/users/07c625af41044628ac45eed1e9ce9a60` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:22.676244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c625af41044628ac45eed1e9ce9a60)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:22.678175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:22.722670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:22.733261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c625af41044628ac45eed1e9ce9a60'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.739916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c625af41044628ac45eed1e9ce9a60'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.751386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c625af41044628ac45eed1e9ce9a60'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.752160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c625af41044628ac45eed1e9ce9a60'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.753115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c625af41044628ac45eed1e9ce9a60'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:22.776537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1117/2268] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:22 2022] DELETE /identity/v3/users/07c625af41044628ac45eed1e9ce9a60 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:23.095478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:23.097544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:23.147330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767a59e-4ead-4035-b070-37a03cca7df2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:23.238256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:23.239372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:23.240311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:23.240311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] PATH_INFO: `/v3/projects/c7fe8b98bf5843319f6cf9bd941ec642` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:23.241563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7fe8b98bf5843319f6cf9bd941ec642)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:23.245511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:23.287379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:23.320839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7fe8b98bf5843319f6cf9bd941ec642', 'request_id': 'req-8d2ef952-d2d0-4928-b720-ed885357201d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.335373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7fe8b98bf5843319f6cf9bd941ec642', 'request_id': 'req-8d2ef952-d2d0-4928-b720-ed885357201d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.350883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7fe8b98bf5843319f6cf9bd941ec642', 'request_id': 'req-8d2ef952-d2d0-4928-b720-ed885357201d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.381006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] The token cache is being invalidate because project c7fe8b98bf5843319f6cf9bd941ec642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:23.383453 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.385701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.391564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1152/2269] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:23 2022] DELETE /identity/v3/projects/c7fe8b98bf5843319f6cf9bd941ec642 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:23.392541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:23.394783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:23.413269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d60e6-a677-4615-87bf-1d4f26d4a07c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:23.481226 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:23.481943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:23.482231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:23.482505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] PATH_INFO: `/v3/users/bccf4e8d12254bd7938ac622006c196f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:23.483835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccf4e8d12254bd7938ac622006c196f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:23.486087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:23.563086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccf4e8d12254bd7938ac622006c196f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.572559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccf4e8d12254bd7938ac622006c196f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.588376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccf4e8d12254bd7938ac622006c196f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.589199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccf4e8d12254bd7938ac622006c196f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.589943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14b8ad5d-9c9e-48e1-8d9e-3066d953ccdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccf4e8d12254bd7938ac622006c196f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:23.609842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1118/2270] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:23 2022] DELETE /identity/v3/users/bccf4e8d12254bd7938ac622006c196f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:23.908149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:23.909694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:23.925207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ef952-d2d0-4928-b720-ed885357201d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.010116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.010903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.011299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.011653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] PATH_INFO: `/v3/projects/5bed5f69c30946ee9fc4bb7a0af4f717` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.012939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bed5f69c30946ee9fc4bb7a0af4f717)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.015000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.066580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bed5f69c30946ee9fc4bb7a0af4f717', 'request_id': 'req-bd39935f-e516-4df4-8d95-2c6a47563e83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:24.073250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bed5f69c30946ee9fc4bb7a0af4f717', 'request_id': 'req-bd39935f-e516-4df4-8d95-2c6a47563e83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:24.079119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bed5f69c30946ee9fc4bb7a0af4f717', 'request_id': 'req-bd39935f-e516-4df4-8d95-2c6a47563e83'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:24.093888 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] The token cache is being invalidate because project 5bed5f69c30946ee9fc4bb7a0af4f717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:24.094110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:24.094884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bd39935f-e516-4df4-8d95-2c6a47563e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:24.096693 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1153/2271] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:23 2022] DELETE /identity/v3/projects/5bed5f69c30946ee9fc4bb7a0af4f717 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:24.101439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dce8c-0df2-4102-965e-cf807090a3b2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.101755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dce8c-0df2-4102-965e-cf807090a3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.102033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dce8c-0df2-4102-965e-cf807090a3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.126245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bc0dce8c-0df2-4102-965e-cf807090a3b2 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:24.127508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bc0dce8c-0df2-4102-965e-cf807090a3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:24.156409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:24.194354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1119/2272] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:24.202066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617eac0e-4955-4389-8338-06506693e4e3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.202388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617eac0e-4955-4389-8338-06506693e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.202733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617eac0e-4955-4389-8338-06506693e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.223544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-617eac0e-4955-4389-8338-06506693e4e3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:24.224924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-617eac0e-4955-4389-8338-06506693e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:24.237789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1154/2273] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:24.239049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dce8c-0df2-4102-965e-cf807090a3b2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.251172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dce8c-0df2-4102-965e-cf807090a3b2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.257396 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.258072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.258356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.258655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.259366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:46:24.259676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.261674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.275242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1120/2274] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:46:24.279564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617eac0e-4955-4389-8338-06506693e4e3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.289667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617eac0e-4955-4389-8338-06506693e4e3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.294699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.295359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.295645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.296103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.296642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.298649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.332419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1155/2275] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:24 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:46:24.334496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.351170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102dfad-6c57-4173-b9f6-9501605df784 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.437191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.437919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.438270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.438606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.439717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.441824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.475167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1121/2276] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:24 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:24.479870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.498454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df850308-7c0a-40f6-b984-6619dfab628f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.504333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.505144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.505437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.505745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.506382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.508232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.522163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1156/2277] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:24.523417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.536972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183b017-e2c5-4a96-a408-e01cda90aaf2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.543926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdf0dd-3eb2-46d4-9c08-3074504c47e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.544659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdf0dd-3eb2-46d4-9c08-3074504c47e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.544950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdf0dd-3eb2-46d4-9c08-3074504c47e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.545382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdf0dd-3eb2-46d4-9c08-3074504c47e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.546065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdf0dd-3eb2-46d4-9c08-3074504c47e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.547701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdf0dd-3eb2-46d4-9c08-3074504c47e5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.566559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1122/2278] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:24.567863 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.579772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3358c-dc06-438a-920e-274d40695743 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.585639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.586487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.586808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.587823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] PATH_INFO: `/v3/projects/0839790e993846da94bce2cae45b13e1/users/54cab5d88195472eb1d4290b7629a8b8/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.602219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:24.607364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0839790e993846da94bce2cae45b13e1, user_id=54cab5d88195472eb1d4290b7629a8b8, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.611185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.640406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1157/2279] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:24 2022] PUT /identity/v3/projects/0839790e993846da94bce2cae45b13e1/users/54cab5d88195472eb1d4290b7629a8b8/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:24.673409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd0b8-8119-43a7-840e-b72ab75c55bc None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.673954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd0b8-8119-43a7-840e-b72ab75c55bc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.673954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd0b8-8119-43a7-840e-b72ab75c55bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.707417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-292fd0b8-8119-43a7-840e-b72ab75c55bc None None] MFA Rules not processed for user `54cab5d88195472eb1d4290b7629a8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:24.709757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-292fd0b8-8119-43a7-840e-b72ab75c55bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:24.831369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1123/2280] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:24.836502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.919923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769971d1-7694-4e5f-ab1a-b537e6c52ac1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:24.924887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:24.925511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:24.925796 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:24.926077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:24.926717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:24.928701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:24.958526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1158/2281] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:24 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:24.963814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd0b8-8119-43a7-840e-b72ab75c55bc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:24.975613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd0b8-8119-43a7-840e-b72ab75c55bc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.002134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:25.041062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.041818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.042161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.042650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.043530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.045550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.079298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1124/2282] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:24 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:25.081398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.099882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9adbaa-1315-4f9b-9569-a92a90c4da32 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.105966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.106680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.107079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.107463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.108239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.110304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.126653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1159/2283] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:25.134746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.156655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b9ad7-f6bc-4e55-893a-f2f21001eea9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.163643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58503a0-52a5-4905-8821-d6defb85a7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.164550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58503a0-52a5-4905-8821-d6defb85a7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.165534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58503a0-52a5-4905-8821-d6defb85a7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.165919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58503a0-52a5-4905-8821-d6defb85a7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.166693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58503a0-52a5-4905-8821-d6defb85a7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.171716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58503a0-52a5-4905-8821-d6defb85a7c9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.197541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1125/2284] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:25.201810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.217531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43a69d-f590-4c10-bf53-b7dd5908520c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.223244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.223686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.223968 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.224243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] PATH_INFO: `/v3/projects/5b5b106815f446f6a625b818690b63eb/users/6e0fd549a91f4d71847926db3866d575/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.241352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5b106815f446f6a625b818690b63eb, user_id=6e0fd549a91f4d71847926db3866d575, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.244816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.263491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1160/2285] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:25 2022] PUT /identity/v3/projects/5b5b106815f446f6a625b818690b63eb/users/6e0fd549a91f4d71847926db3866d575/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:25.274398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3342-e175-4c0c-8e9d-15449fae013c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.274768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3342-e175-4c0c-8e9d-15449fae013c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.275062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3342-e175-4c0c-8e9d-15449fae013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.307052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-10fc3342-e175-4c0c-8e9d-15449fae013c None None] MFA Rules not processed for user `6e0fd549a91f4d71847926db3866d575`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:25.308335 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-10fc3342-e175-4c0c-8e9d-15449fae013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:25.380479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1126/2286] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:25.385449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.454316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90a068-9963-4385-b55d-c317a7f252f2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.459763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.460526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.460895 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.461306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.462279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.464400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.499578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1161/2287] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:46:25 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:46:25.504388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc3342-e175-4c0c-8e9d-15449fae013c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.517218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc3342-e175-4c0c-8e9d-15449fae013c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.589056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.589791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.590149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.590500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.591238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.596456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.620572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1127/2288] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:46:25 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:46:25.625035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.637272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbf283-050b-446b-9ba1-6a1265a2291e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.651997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.653087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.653448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.653847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.654609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.656405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.673240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1162/2289] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:25.674438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.701607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88310d1d-924b-4934-9583-0e809cfccc1a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.707035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.707911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.708198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.708555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] PATH_INFO: `/v3/projects/99a1a09d556846339dd5d37d3323ae4a/users/7bd4cbc58c4a408a89211231d7818660/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.746047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a1a09d556846339dd5d37d3323ae4a, user_id=7bd4cbc58c4a408a89211231d7818660, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.753994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.781180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:25.786544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1128/2290] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:25 2022] PUT /identity/v3/projects/99a1a09d556846339dd5d37d3323ae4a/users/7bd4cbc58c4a408a89211231d7818660/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:46:25.791434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.804520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c955b-75c3-442b-a697-9d6a99b88d34 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.814741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:25.903263 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43c62e-7858-4fc2-bbae-c70035c2ec3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.904337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43c62e-7858-4fc2-bbae-c70035c2ec3f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.904789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43c62e-7858-4fc2-bbae-c70035c2ec3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.905221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43c62e-7858-4fc2-bbae-c70035c2ec3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.906056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43c62e-7858-4fc2-bbae-c70035c2ec3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.907919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43c62e-7858-4fc2-bbae-c70035c2ec3f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.925440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1163/2291] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:46:25.929610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:25.942604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96beafb-08b0-4e16-a340-f04403b6a005 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:25.947832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:25.948659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.949050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.949430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] PATH_INFO: `/v3/projects/99a1a09d556846339dd5d37d3323ae4a/users/7bd4cbc58c4a408a89211231d7818660/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:25.951588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a1a09d556846339dd5d37d3323ae4a, user_id=7bd4cbc58c4a408a89211231d7818660, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:25.954806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:25.976223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1129/2292] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:46:25 2022] PUT /identity/v3/projects/99a1a09d556846339dd5d37d3323ae4a/users/7bd4cbc58c4a408a89211231d7818660/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:25.987805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:25.988186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:25.988547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:26.023041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] MFA Rules not processed for user `7bd4cbc58c4a408a89211231d7818660`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:26.024582 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:26.144923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1164/2293] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:26.157787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:26.159643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:26.272448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074a270-cdf6-4d81-a619-b00c4973caf4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:26.277132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:26.277808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:26.278076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:26.278352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:26.344600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:26.346161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:26.357749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1130/2294] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:27.932128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:27.933620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:27.946446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c61bc6-f645-438e-955c-2dae22f78a4c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:27.950415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:27.951038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:27.951414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:27.951633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:28.020945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:28.022421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:28.033708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1165/2295] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:28.702246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:28.714447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396fa6-d22f-4665-8d41-f8842d784d89 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:28.730549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b056d-b946-44ae-bb97-085e71df5bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:28.730549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b056d-b946-44ae-bb97-085e71df5bbe service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:28.730549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b056d-b946-44ae-bb97-085e71df5bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:28.730549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b056d-b946-44ae-bb97-085e71df5bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:28.747835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b056d-b946-44ae-bb97-085e71df5bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:28.749873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b056d-b946-44ae-bb97-085e71df5bbe service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:28.772788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1131/2296] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:32.798627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:32.838691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:32.882397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dceb09-2608-48fd-a6ef-6321fec95a38 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:32.890232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:32.891162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:32.891661 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:32.892080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] PATH_INFO: `/v3/users/54cab5d88195472eb1d4290b7629a8b8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:32.894044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cab5d88195472eb1d4290b7629a8b8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:32.897175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:32.956238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cab5d88195472eb1d4290b7629a8b8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:32.956910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cab5d88195472eb1d4290b7629a8b8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:32.964224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cab5d88195472eb1d4290b7629a8b8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:32.964871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cab5d88195472eb1d4290b7629a8b8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:32.974159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cab5d88195472eb1d4290b7629a8b8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:32.992752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1166/2297] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:32 2022] DELETE /identity/v3/users/54cab5d88195472eb1d4290b7629a8b8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:32.997467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:32.997970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:32.998087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:33.018124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:33.019331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:33.080467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1132/2298] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:33.091260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:33.093590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:33.161904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab902e8-6183-4706-88af-cdbe29752dd3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:33.165928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:33.166671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:33.166956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:33.167235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:33.176678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:33.178121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:33.188926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1167/2299] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:33.469623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:33.471569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:33.490843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad0a4c-f5eb-495a-80f8-9297384f6ff2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:33.504003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:33.504806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:33.505191 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:33.505504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] PATH_INFO: `/v3/projects/0839790e993846da94bce2cae45b13e1` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:33.506868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0839790e993846da94bce2cae45b13e1)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:33.509212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:33.539618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:33.568819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0839790e993846da94bce2cae45b13e1', 'request_id': 'req-cc2161ab-8ccf-40db-a11d-113ae51c252f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.575506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0839790e993846da94bce2cae45b13e1', 'request_id': 'req-cc2161ab-8ccf-40db-a11d-113ae51c252f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.581897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0839790e993846da94bce2cae45b13e1', 'request_id': 'req-cc2161ab-8ccf-40db-a11d-113ae51c252f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.596216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] The token cache is being invalidate because project 0839790e993846da94bce2cae45b13e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:33.596428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.597041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.598581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1133/2300] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:33 2022] DELETE /identity/v3/projects/0839790e993846da94bce2cae45b13e1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:33.602659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:33.613177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6e22f-7791-4ca6-a3e8-80f8749ec544 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:33.664974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:33.675389 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:33.676370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:33.676766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:33.677289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] PATH_INFO: `/v3/users/6e0fd549a91f4d71847926db3866d575` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:33.678534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0fd549a91f4d71847926db3866d575)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:33.680715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:33.746878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0fd549a91f4d71847926db3866d575'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.747672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0fd549a91f4d71847926db3866d575'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.759068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0fd549a91f4d71847926db3866d575'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.759875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0fd549a91f4d71847926db3866d575'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.770992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0fd549a91f4d71847926db3866d575'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:33.796071 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1168/2301] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:33 2022] DELETE /identity/v3/users/6e0fd549a91f4d71847926db3866d575 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:34.400734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:34.403502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:34.419498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2161ab-8ccf-40db-a11d-113ae51c252f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:34.482897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:34.483544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:34.483838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:34.484118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] PATH_INFO: `/v3/projects/5b5b106815f446f6a625b818690b63eb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:34.485268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5b106815f446f6a625b818690b63eb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:34.487228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:34.539083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5b106815f446f6a625b818690b63eb', 'request_id': 'req-231bb990-09b7-49a9-8e60-5fae35bbc755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.545072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5b106815f446f6a625b818690b63eb', 'request_id': 'req-231bb990-09b7-49a9-8e60-5fae35bbc755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.551596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5b106815f446f6a625b818690b63eb', 'request_id': 'req-231bb990-09b7-49a9-8e60-5fae35bbc755'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.565588 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] The token cache is being invalidate because project 5b5b106815f446f6a625b818690b63eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:34.565836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.566477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.568280 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1134/2302] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:34 2022] DELETE /identity/v3/projects/5b5b106815f446f6a625b818690b63eb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:46:34.572281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:34.573835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:34.586636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb0fdb-628c-4631-aa8b-7275723e4fef admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:34.646583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:34.647415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:34.647572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:34.647823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] PATH_INFO: `/v3/users/7bd4cbc58c4a408a89211231d7818660` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:34.648965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd4cbc58c4a408a89211231d7818660)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:34.651185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:34.700332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd4cbc58c4a408a89211231d7818660'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.701447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd4cbc58c4a408a89211231d7818660'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.710533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd4cbc58c4a408a89211231d7818660'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.711314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd4cbc58c4a408a89211231d7818660'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.720797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd4cbc58c4a408a89211231d7818660'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:34.736255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:34.753116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1169/2303] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:46:34 2022] DELETE /identity/v3/users/7bd4cbc58c4a408a89211231d7818660 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:46:35.171659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:35.173288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:35.246991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:35.249166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:35.289803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:35.322344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bb990-09b7-49a9-8e60-5fae35bbc755 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:35.398028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c57ad8-b2f1-4a8d-a735-6d2934a1dcb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:35.399014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c57ad8-b2f1-4a8d-a735-6d2934a1dcb9 service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:35.399497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c57ad8-b2f1-4a8d-a735-6d2934a1dcb9 service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:35.399940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c57ad8-b2f1-4a8d-a735-6d2934a1dcb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:35.401913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308138b-5a64-4437-9b80-d04ec7ce45e5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:35.402989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d4c57ad8-b2f1-4a8d-a735-6d2934a1dcb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:35.408622 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:35.409444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:35.409965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:35.410341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] PATH_INFO: `/v3/projects/99a1a09d556846339dd5d37d3323ae4a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:35.411755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a1a09d556846339dd5d37d3323ae4a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:35.414075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:35.420122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c57ad8-b2f1-4a8d-a735-6d2934a1dcb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:35.424097 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c57ad8-b2f1-4a8d-a735-6d2934a1dcb9 service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:35.450663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1135/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:35.489917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a1a09d556846339dd5d37d3323ae4a', 'request_id': 'req-95b6509a-127d-413a-8a89-ad63201d7fc6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:35.500200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a1a09d556846339dd5d37d3323ae4a', 'request_id': 'req-95b6509a-127d-413a-8a89-ad63201d7fc6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:35.511555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a1a09d556846339dd5d37d3323ae4a', 'request_id': 'req-95b6509a-127d-413a-8a89-ad63201d7fc6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:35.543078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] The token cache is being invalidate because project 99a1a09d556846339dd5d37d3323ae4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:46:35.543473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:35.544116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:46:35.559225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1170/2305] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:46:35 2022] DELETE /identity/v3/projects/99a1a09d556846339dd5d37d3323ae4a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:46:45.431244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=39 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:45.537452 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=54,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:51.042791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:51.043326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:51.043577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:51.065278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:46:51.066510 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:51.130003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1136/2306] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 06:46:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:46:51.158462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:51.160027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:51.246740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6509a-127d-413a-8a89-ad63201d7fc6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:51.252297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:51.252910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:51.253191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:51.253470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:51.262045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:51.263698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:51.271847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1171/2307] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:46:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:47:01.119173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=25 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:01.268636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=32 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:24.761969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:24.764116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:24.833073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee4d99-254b-4ae3-b309-bacc831baa83 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:24.838358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:24.839124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:24.839431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:24.839702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] PATH_INFO: `/v3/users/686bbf803837400e94f1df1efd04a407` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:24.840861 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=686bbf803837400e94f1df1efd04a407)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:24.842886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:24.894535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686bbf803837400e94f1df1efd04a407'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:24.902263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686bbf803837400e94f1df1efd04a407'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:24.912759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686bbf803837400e94f1df1efd04a407'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:24.913520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686bbf803837400e94f1df1efd04a407'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:24.914069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-558f0eb8-dae3-4aad-9e29-6c24019e6c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686bbf803837400e94f1df1efd04a407'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:24.932316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1137/2308] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:47:24 2022] DELETE /identity/v3/users/686bbf803837400e94f1df1efd04a407 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:25.125653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.138431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52d10a-627e-4316-8674-05d44ddedfac service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.209508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.210173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.210913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.211186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] PATH_INFO: `/v3/projects/b1d68e8e7e104bccb721e3c036a7907a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.212579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d68e8e7e104bccb721e3c036a7907a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.214560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:25.282612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d68e8e7e104bccb721e3c036a7907a', 'request_id': 'req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:25.290629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d68e8e7e104bccb721e3c036a7907a', 'request_id': 'req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:25.296402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d68e8e7e104bccb721e3c036a7907a', 'request_id': 'req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:25.311760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] The token cache is being invalidate because project b1d68e8e7e104bccb721e3c036a7907a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:47:25.312049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:25.312700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cc4722bc-2f96-44c4-b707-5bbcdab4f11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:25.314247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1172/2309] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:47:25 2022] DELETE /identity/v3/projects/b1d68e8e7e104bccb721e3c036a7907a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:25.319433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7981c60-69f7-4645-a467-6cc251283f8d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.319797 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7981c60-69f7-4645-a467-6cc251283f8d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.320733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7981c60-69f7-4645-a467-6cc251283f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.340045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d7981c60-69f7-4645-a467-6cc251283f8d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:25.341295 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d7981c60-69f7-4645-a467-6cc251283f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:25.408240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1138/2310] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:25.417040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504a0bc-3b4e-47f8-9a6f-f199b59ef1c6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.417472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504a0bc-3b4e-47f8-9a6f-f199b59ef1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.417802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504a0bc-3b4e-47f8-9a6f-f199b59ef1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.439733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7504a0bc-3b4e-47f8-9a6f-f199b59ef1c6 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:25.440974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7504a0bc-3b4e-47f8-9a6f-f199b59ef1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:25.449813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1173/2311] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:25.454366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7981c60-69f7-4645-a467-6cc251283f8d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.465516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7981c60-69f7-4645-a467-6cc251283f8d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.471393 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.471993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.472293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.473025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.474044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:47:25.474362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.476051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:25.489677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1139/2312] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:47:25.493951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504a0bc-3b4e-47f8-9a6f-f199b59ef1c6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.505104 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504a0bc-3b4e-47f8-9a6f-f199b59ef1c6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.509956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.510571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.510854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.511131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.511827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.513772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:25.545944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1174/2313] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:47:25 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:47:25.550187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.562843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7fe17-cb57-4ec8-b672-a8ccafa9fa47 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.567494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:25.622734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.623244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.623556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.623813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.624441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.626475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:25.649538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1140/2314] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:47:25 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:47:25.653753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.665049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a5b05-1f7d-40c6-a8c2-381910478a2c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.670385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.671009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.671327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.671604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.672268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.673901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:25.685450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1175/2315] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:47:25.690199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.703455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632c641-2114-41db-81cc-e95a1267d576 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.709609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17dbdf-d7d9-4f58-ac89-e63bc086d9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.710225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dbdf-d7d9-4f58-ac89-e63bc086d9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.710511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dbdf-d7d9-4f58-ac89-e63bc086d9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.710785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dbdf-d7d9-4f58-ac89-e63bc086d9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.711493 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17dbdf-d7d9-4f58-ac89-e63bc086d9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.713118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17dbdf-d7d9-4f58-ac89-e63bc086d9e2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:25.725501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1141/2316] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:47:25.729881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.741111 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a8894-7fb5-4970-9458-3170a3e8d77b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.746547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.747399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.747807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.748168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] PATH_INFO: `/v3/projects/f183480b0a0245bdb066e6233f71f129/users/b0ad91c12a6449bfb64b9dde6e152a5c/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.764013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f183480b0a0245bdb066e6233f71f129, user_id=b0ad91c12a6449bfb64b9dde6e152a5c, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.766678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:25.781556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1176/2317] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:47:25 2022] PUT /identity/v3/projects/f183480b0a0245bdb066e6233f71f129/users/b0ad91c12a6449bfb64b9dde6e152a5c/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:25.793155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b44ae-e3d1-49ec-bed6-b3707bae7720 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.793694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b44ae-e3d1-49ec-bed6-b3707bae7720 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.793694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b44ae-e3d1-49ec-bed6-b3707bae7720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.825446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4f0b44ae-e3d1-49ec-bed6-b3707bae7720 None None] MFA Rules not processed for user `b0ad91c12a6449bfb64b9dde6e152a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:25.826834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4f0b44ae-e3d1-49ec-bed6-b3707bae7720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:25.898471 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1142/2318] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:47:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:25.909971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:25.912495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:25.947203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:25.980794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada3c2b-fdfe-429e-bfbd-e335b7ba6ccc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:25.984578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:25.985207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:25.985498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:25.985805 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:25.994828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:25.996330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:26.006132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1177/2319] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:26.290342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cd1ea-02b1-4deb-86d1-c5ea85b2d3d6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:26.290835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cd1ea-02b1-4deb-86d1-c5ea85b2d3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:26.290924 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cd1ea-02b1-4deb-86d1-c5ea85b2d3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:26.318255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-588cd1ea-02b1-4deb-86d1-c5ea85b2d3d6 None None] MFA Rules not processed for user `f264e354671d4456bb33668fb11d9636`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:26.319827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-588cd1ea-02b1-4deb-86d1-c5ea85b2d3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:26.378397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1143/2320] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:47:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:36.001991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=14 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:36.371725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:42.582623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:42.660210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54652356-3088-4aa3-b425-e7493ccaea14 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:42.665217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:42.665857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:42.666145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:42.666424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] PATH_INFO: `/v3/users/b0ad91c12a6449bfb64b9dde6e152a5c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:42.667725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ad91c12a6449bfb64b9dde6e152a5c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:42.669913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:42.732850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ad91c12a6449bfb64b9dde6e152a5c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:42.733581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ad91c12a6449bfb64b9dde6e152a5c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:42.743443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ad91c12a6449bfb64b9dde6e152a5c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:42.744277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ad91c12a6449bfb64b9dde6e152a5c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:42.753359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ad91c12a6449bfb64b9dde6e152a5c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:42.771795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1178/2321] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:47:42 2022] DELETE /identity/v3/users/b0ad91c12a6449bfb64b9dde6e152a5c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:42.776714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:42.777288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:42.777388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:42.797446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:42.800157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:42.865692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1144/2322] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:42.876548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:42.878110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:42.937811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040f5ca-e70a-4985-8e70-1e5a22f7a642 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:42.941481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05946fa5-ed8a-4020-b0d9-e72ea8697cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:42.942113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05946fa5-ed8a-4020-b0d9-e72ea8697cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:42.942370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05946fa5-ed8a-4020-b0d9-e72ea8697cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:42.942640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05946fa5-ed8a-4020-b0d9-e72ea8697cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:42.953639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05946fa5-ed8a-4020-b0d9-e72ea8697cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:42.955219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05946fa5-ed8a-4020-b0d9-e72ea8697cd1 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:42.967296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1179/2323] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:43.274807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:43.276402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:43.288837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96678bc4-0b66-40a1-a83d-5cf6cfa933cd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:43.294232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:43.294902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:43.295149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:43.295455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] PATH_INFO: `/v3/projects/f183480b0a0245bdb066e6233f71f129` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:43.296654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f183480b0a0245bdb066e6233f71f129)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:43.298587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:43.347338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f183480b0a0245bdb066e6233f71f129', 'request_id': 'req-bfc0ac3c-8549-4329-aac6-c9ded8569f15'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:43.353178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f183480b0a0245bdb066e6233f71f129', 'request_id': 'req-bfc0ac3c-8549-4329-aac6-c9ded8569f15'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:43.359591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f183480b0a0245bdb066e6233f71f129', 'request_id': 'req-bfc0ac3c-8549-4329-aac6-c9ded8569f15'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:43.374985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] The token cache is being invalidate because project f183480b0a0245bdb066e6233f71f129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:47:43.375374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:43.376086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfc0ac3c-8549-4329-aac6-c9ded8569f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:43.377628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1145/2324] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:47:43 2022] DELETE /identity/v3/projects/f183480b0a0245bdb066e6233f71f129 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:51.806033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:51.806538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:51.806634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:51.826736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] MFA Rules not processed for user `bfb5dc1fa16f465ba1ebca000b665d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:51.827971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:51.847769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:51.887250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1180/2325] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:47:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:53.372222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:01.250224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:01.250909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:01.250909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:01.270435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] MFA Rules not processed for user `2c917c21c7974c8d9746e8b05d9c8b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:01.271673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:01.337536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1146/2326] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:48:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:01.880911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=15 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:11.325564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=25 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:28.923611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:28.925275 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:29.020987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338199be-fc07-4adb-82e5-6f58848e4ca6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:29.028573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:29.029190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:29.029467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:29.029735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:29.031859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:29.100386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:29.102004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:29.112037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1181/2327] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:29.893175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:29.894610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:29.955719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7236c3-ec27-4f42-8a90-4f6f84933426 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:29.959768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:29.960378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:29.960665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:29.960940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:29.963100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:29.977117 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:29.978636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:29.989598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1147/2328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:30.578089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:30.579780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:30.644002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b831e2-8708-4196-bdc8-63d313349359 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:30.648516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0da2d-216c-4253-8597-bea1f3b4b4c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:30.649367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0da2d-216c-4253-8597-bea1f3b4b4c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:30.649657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0da2d-216c-4253-8597-bea1f3b4b4c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:30.649929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0da2d-216c-4253-8597-bea1f3b4b4c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:30.652048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f5a0da2d-216c-4253-8597-bea1f3b4b4c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:30.666376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0da2d-216c-4253-8597-bea1f3b4b4c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:30.667818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0da2d-216c-4253-8597-bea1f3b4b4c0 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:30.678755 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1182/2329] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:33.489048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:33.491364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:33.507310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669ca3-1d02-4252-984a-53dafa89bd47 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:33.514055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:33.516777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:33.517071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:33.517303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:33.522798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:33.601072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:33.603021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:33.613501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1148/2330] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:35.892928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:35.893460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:35.893576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:35.913698 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] MFA Rules not processed for user `1d26cf886087408097cc2e435a785454`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:35.914918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:35.923966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1183/2331] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:48:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:36.251507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:36.261392 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb4254-bae1-4081-af76-77dc3629c6dc service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:36.265412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:36.266021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:36.266596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:36.266873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:36.268826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:36.279151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:36.280608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:36.289159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1149/2332] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:36.620526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:36.622191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:36.662166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:36.698807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439db0c-afd7-4159-a639-fde3daec2e76 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:36.704472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:36.705072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:36.705354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:36.705677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:36.707523 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:36.719599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:36.721217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:36.731876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1184/2333] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:41.596910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:41.598561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:41.609936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5f888-0cdb-497b-83a4-75a0ad7b31f5 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:41.614568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:41.615229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:41.615484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:41.615985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:41.618048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:41.629084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:41.630539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:41.638856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1150/2334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:46.727149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=17 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:47.612371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.623168 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85069cb6-ed3d-40cd-98d7-365455ffb9ba service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.628460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6250b-e686-4bbb-813c-4631548c3975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.629074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6250b-e686-4bbb-813c-4631548c3975 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.629376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6250b-e686-4bbb-813c-4631548c3975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.629658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6250b-e686-4bbb-813c-4631548c3975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.632207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6df6250b-e686-4bbb-813c-4631548c3975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:47.646894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6250b-e686-4bbb-813c-4631548c3975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.648418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6250b-e686-4bbb-813c-4631548c3975 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.658281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1185/2335] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:49.910925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:49.925306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb16aa-1293-483c-9102-139b1ca8c0cf service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:49.931330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:49.932018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:49.932299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:49.932578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:49.934425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:49.947745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:49.949324 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:49.960086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1151/2336] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:50.291246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b996b1-a56b-4065-9818-89be6b4269eb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:50.292228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b996b1-a56b-4065-9818-89be6b4269eb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:50.292472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b996b1-a56b-4065-9818-89be6b4269eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:50.304576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:50.319328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5224-3b0f-407c-8a1d-0bb3f7a07e6c service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:50.326911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e4a50-c21c-49f6-9569-c5c26f714e24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:50.327897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4a50-c21c-49f6-9569-c5c26f714e24 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:50.329148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4a50-c21c-49f6-9569-c5c26f714e24 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:50.329641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4a50-c21c-49f6-9569-c5c26f714e24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:50.331866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-357e4a50-c21c-49f6-9569-c5c26f714e24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:50.333497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-57b996b1-a56b-4065-9818-89be6b4269eb None None] MFA Rules not processed for user `f264e354671d4456bb33668fb11d9636`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:50.335072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-57b996b1-a56b-4065-9818-89be6b4269eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:50.344350 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1186/2337] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:48:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:50.350445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e4a50-c21c-49f6-9569-c5c26f714e24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:50.352461 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e4a50-c21c-49f6-9569-c5c26f714e24 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:50.365232 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1152/2338] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:56.625912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a363b-6c98-49e1-9589-321a2618f382 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:56.626570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a363b-6c98-49e1-9589-321a2618f382 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:56.626729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a363b-6c98-49e1-9589-321a2618f382 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:56.630737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1187/2339] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:48:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:48:56.640725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a78a3-bc16-4406-b5ca-c8d397d60892 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:56.641338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a78a3-bc16-4406-b5ca-c8d397d60892 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:56.643731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a78a3-bc16-4406-b5ca-c8d397d60892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:56.652839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=79 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:56.667856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e42a78a3-bc16-4406-b5ca-c8d397d60892 None None] MFA Rules not processed for user `2fc4592ef54b41059fa7767a5ca3aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:56.670752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e42a78a3-bc16-4406-b5ca-c8d397d60892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:56.678608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1153/2340] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:48:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:56.691706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a363b-6c98-49e1-9589-321a2618f382 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:56.703818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a363b-6c98-49e1-9589-321a2618f382 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:56.707619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574241cb-dfd4-4ecc-8d08-b5893e199378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:56.708233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574241cb-dfd4-4ecc-8d08-b5893e199378 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:56.708513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574241cb-dfd4-4ecc-8d08-b5893e199378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:56.708788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574241cb-dfd4-4ecc-8d08-b5893e199378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:56.717807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574241cb-dfd4-4ecc-8d08-b5893e199378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:56.720515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574241cb-dfd4-4ecc-8d08-b5893e199378 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:56.729317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1188/2341] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:56.835183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4d11f-4a5c-4bb8-b550-7bd22dde9aa3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:56.835183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4d11f-4a5c-4bb8-b550-7bd22dde9aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:56.835183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4d11f-4a5c-4bb8-b550-7bd22dde9aa3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:56.835183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1154/2342] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:48:56.836356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd6924-6d8c-48b0-8b13-a865bf983eaf None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:56.836648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd6924-6d8c-48b0-8b13-a865bf983eaf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:56.836958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd6924-6d8c-48b0-8b13-a865bf983eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:56.858037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-25cd6924-6d8c-48b0-8b13-a865bf983eaf None None] MFA Rules not processed for user `8e1ec869263246f58f5a32d855a3d302`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:56.859730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-25cd6924-6d8c-48b0-8b13-a865bf983eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:56.869250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1189/2343] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:48:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:56.880714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4d11f-4a5c-4bb8-b550-7bd22dde9aa3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:56.898286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4d11f-4a5c-4bb8-b550-7bd22dde9aa3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:56.904991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:56.905917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:56.906173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:56.906489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:56.918135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:56.919734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:56.930981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1155/2344] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:59.414291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd6924-6d8c-48b0-8b13-a865bf983eaf None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:59.428455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd6924-6d8c-48b0-8b13-a865bf983eaf None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:59.434957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67de1e4-8b4e-46d7-a1ce-06b8882607fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:59.435967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de1e4-8b4e-46d7-a1ce-06b8882607fc service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:59.436247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de1e4-8b4e-46d7-a1ce-06b8882607fc service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:59.436518 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de1e4-8b4e-46d7-a1ce-06b8882607fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:59.438768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c67de1e4-8b4e-46d7-a1ce-06b8882607fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:59.450912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67de1e4-8b4e-46d7-a1ce-06b8882607fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:59.452613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67de1e4-8b4e-46d7-a1ce-06b8882607fc service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:59.463172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1190/2345] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:06.926252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=8 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:08.828769 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:08.837749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f73eee-4ee3-4d8b-b9a1-47c3efea83b4 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:08.841942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:08.842544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:08.842825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:08.843094 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:08.845342 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:08.913760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:08.915084 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:08.925302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1156/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:09.458775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=24 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:10.940823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:10.941513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:10.941513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:10.968428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:10.970422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:11.056390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1191/2347] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:49:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:11.106943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:11.110454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:11.187981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d583559-4049-43a2-9c33-46c137626337 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:11.194950 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:11.195615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:11.196061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:11.196497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:11.207255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:11.209032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:11.239466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1157/2348] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:49:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:49:12.867503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:12.869586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:12.885488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d8e55-6efb-4245-95f1-231bd76dfdaf None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:12.892734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:12.893626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:12.894133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:12.894602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:12.896413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:12.914903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:12.916585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:12.929766 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1192/2349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:12.945370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:12.946868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:12.970120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fcf55-3897-4779-a493-bac6d95c290c service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:12.975447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:12.976076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:12.976357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:12.976630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] PATH_INFO: `/v3/users/2c917c21c7974c8d9746e8b05d9c8b9f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:12.977881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c917c21c7974c8d9746e8b05d9c8b9f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:12.980050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:13.050429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c917c21c7974c8d9746e8b05d9c8b9f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.059905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c917c21c7974c8d9746e8b05d9c8b9f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.071038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c917c21c7974c8d9746e8b05d9c8b9f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.071458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c917c21c7974c8d9746e8b05d9c8b9f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.072243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c168dd4-2d00-4d3d-87c6-2a1c5415f96d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c917c21c7974c8d9746e8b05d9c8b9f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.099344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:13.115222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1158/2350] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:12 2022] DELETE /identity/v3/users/2c917c21c7974c8d9746e8b05d9c8b9f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:13.336348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:13.357567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca48d-efb2-46ce-b2e0-15027d76330a service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:13.449072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:13.449791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:13.450137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:13.450469 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] PATH_INFO: `/v3/projects/2a401374685240dc94fe28ca99ad4e11` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:13.451830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a401374685240dc94fe28ca99ad4e11)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:13.454690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:13.514656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a401374685240dc94fe28ca99ad4e11', 'request_id': 'req-31559182-e198-4c45-8e37-ff6ac7ec352e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.521951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a401374685240dc94fe28ca99ad4e11', 'request_id': 'req-31559182-e198-4c45-8e37-ff6ac7ec352e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.530347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a401374685240dc94fe28ca99ad4e11', 'request_id': 'req-31559182-e198-4c45-8e37-ff6ac7ec352e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.549195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] The token cache is being invalidate because project 2a401374685240dc94fe28ca99ad4e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:13.549498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.550299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-31559182-e198-4c45-8e37-ff6ac7ec352e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:13.552213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1193/2351] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:13 2022] DELETE /identity/v3/projects/2a401374685240dc94fe28ca99ad4e11 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:13.557022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c83ae8-fceb-4758-aa9e-1b5ac18675e2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:13.557317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c83ae8-fceb-4758-aa9e-1b5ac18675e2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:13.557586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c83ae8-fceb-4758-aa9e-1b5ac18675e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:13.582687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-40c83ae8-fceb-4758-aa9e-1b5ac18675e2 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:13.584228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-40c83ae8-fceb-4758-aa9e-1b5ac18675e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:13.674844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1159/2352] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:13.675877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70895c1b-d754-4506-ae39-6be0cba43827 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:13.676203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70895c1b-d754-4506-ae39-6be0cba43827 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:13.676509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70895c1b-d754-4506-ae39-6be0cba43827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:13.724696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-70895c1b-d754-4506-ae39-6be0cba43827 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:13.726123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-70895c1b-d754-4506-ae39-6be0cba43827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:13.748988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1194/2353] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:13.751961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c83ae8-fceb-4758-aa9e-1b5ac18675e2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:13.765195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c83ae8-fceb-4758-aa9e-1b5ac18675e2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:13.778088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:13.778873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:13.779205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:13.779492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:13.780160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:13.780470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:13.782397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:13.805836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1160/2354] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:13.810639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70895c1b-d754-4506-ae39-6be0cba43827 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:13.821151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70895c1b-d754-4506-ae39-6be0cba43827 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:13.826331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:13.826944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:13.827258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:13.827533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:13.828402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:13.831163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:13.849823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:13.874689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1195/2355] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:13 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:49:13.876677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:13.899696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9311724-64bd-42ff-a888-9db2575bda26 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:13.984504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:13.986052 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:13.986538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:13.987057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:13.987896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:13.990157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.019238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1161/2356] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:13 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:49:14.021211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.033802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9bbe-56ef-42f7-8cca-4f4a86cf9e54 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.038722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.039686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.040149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.040808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.041752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.044336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.057175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1196/2357] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:14.061703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.076673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b2520-80e7-40bb-815e-14661fd9c1b4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.083085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422de2f-dad3-4928-9bce-fd7f9bf8b841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.083659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422de2f-dad3-4928-9bce-fd7f9bf8b841 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.083937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422de2f-dad3-4928-9bce-fd7f9bf8b841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.084212 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422de2f-dad3-4928-9bce-fd7f9bf8b841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.084880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422de2f-dad3-4928-9bce-fd7f9bf8b841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.086523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422de2f-dad3-4928-9bce-fd7f9bf8b841 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.100169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1162/2358] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:14.104425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.116651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15ac2c-ba17-4a4a-978f-d6dd70558fc4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.124480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.125494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.125777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.126048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] PATH_INFO: `/v3/projects/7a89a9ded1fd41b385fc1ce46d74893d/users/b0065f5c10bf462999dd554f182775a3/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.146438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a89a9ded1fd41b385fc1ce46d74893d, user_id=b0065f5c10bf462999dd554f182775a3, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.149214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.171439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1197/2359] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:14 2022] PUT /identity/v3/projects/7a89a9ded1fd41b385fc1ce46d74893d/users/b0065f5c10bf462999dd554f182775a3/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:14.180463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae5c5-6dbc-4ec4-9957-6a8b4ed8bb0e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.180463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae5c5-6dbc-4ec4-9957-6a8b4ed8bb0e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.180871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae5c5-6dbc-4ec4-9957-6a8b4ed8bb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.230065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1bfae5c5-6dbc-4ec4-9957-6a8b4ed8bb0e None None] MFA Rules not processed for user `b0065f5c10bf462999dd554f182775a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:14.231327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1bfae5c5-6dbc-4ec4-9957-6a8b4ed8bb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:14.313784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:14.337017 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1163/2360] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:14.341395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.473096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ed2fc-667d-4b4b-b441-a8a694861634 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.478884 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.480928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.481402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.482098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.483377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.485809 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.522751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1198/2361] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:14 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:14.527630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfae5c5-6dbc-4ec4-9957-6a8b4ed8bb0e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.541775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfae5c5-6dbc-4ec4-9957-6a8b4ed8bb0e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.608538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.608538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.608538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.609453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.609453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.611729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.636469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1164/2362] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:14 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:14.640502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.653235 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3555c-8736-4b6f-8d44-59af52b9d8e3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.658234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.658929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.659318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.659679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.660401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.664324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.676225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1199/2363] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:14.680684 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.692011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11db00-2dd2-4290-ab70-22283da96125 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.696831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.697627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.698009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.698351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] PATH_INFO: `/v3/projects/ce3bc99858e54ca3a7846caa09182bc2/users/dfd770292b4041bb9bb9363f989bf9c8/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.716032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce3bc99858e54ca3a7846caa09182bc2, user_id=dfd770292b4041bb9bb9363f989bf9c8, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.719115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.741398 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1165/2364] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:14 2022] PUT /identity/v3/projects/ce3bc99858e54ca3a7846caa09182bc2/users/dfd770292b4041bb9bb9363f989bf9c8/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:14.742682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.758368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbc4d1-8225-4da9-a9cb-d33a6bd0ecd6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.821770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb727ef4-7d3d-4023-b0db-999b189ada75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.822458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb727ef4-7d3d-4023-b0db-999b189ada75 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.822749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb727ef4-7d3d-4023-b0db-999b189ada75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.823179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb727ef4-7d3d-4023-b0db-999b189ada75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.823883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb727ef4-7d3d-4023-b0db-999b189ada75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.825526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb727ef4-7d3d-4023-b0db-999b189ada75 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.840729 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1200/2365] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:14.842244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:14.854118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b930f7-fea4-4154-b28d-35e842ea8156 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:14.861044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:14.861732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.862011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.862285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] PATH_INFO: `/v3/projects/ce3bc99858e54ca3a7846caa09182bc2/users/dfd770292b4041bb9bb9363f989bf9c8/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.864261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce3bc99858e54ca3a7846caa09182bc2, user_id=dfd770292b4041bb9bb9363f989bf9c8, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:14.866935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:14.878988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1166/2366] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:14 2022] PUT /identity/v3/projects/ce3bc99858e54ca3a7846caa09182bc2/users/dfd770292b4041bb9bb9363f989bf9c8/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:14.889424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db5dcf-e59e-47cd-a1d7-253dc8f9a339 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:14.889989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db5dcf-e59e-47cd-a1d7-253dc8f9a339 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:14.890139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db5dcf-e59e-47cd-a1d7-253dc8f9a339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:14.925484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-12db5dcf-e59e-47cd-a1d7-253dc8f9a339 None None] MFA Rules not processed for user `dfd770292b4041bb9bb9363f989bf9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:14.926775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-12db5dcf-e59e-47cd-a1d7-253dc8f9a339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:14.957793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:15.026731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1201/2367] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:15.050296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:15.052245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:15.119540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a496c8-f9c4-4327-9ec3-f5ef58a1c2cb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:15.131466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:15.131466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:15.131466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:15.131466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:15.142128 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:15.143859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:15.156632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1167/2368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:15.601478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db5dcf-e59e-47cd-a1d7-253dc8f9a339 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:15.615678 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db5dcf-e59e-47cd-a1d7-253dc8f9a339 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:15.623059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd770292b4041bb9bb9363f989bf9c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce3bc99858e54ca3a7846caa09182bc2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:15.623980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:15.624193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:15.624519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:15.625437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:15.625816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:15.627490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:15.646407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1202/2369] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:15.647418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:15.659186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117f7c0-bec5-46e7-9a12-5866eae1b19c service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:15.664492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd770292b4041bb9bb9363f989bf9c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce3bc99858e54ca3a7846caa09182bc2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:15.665184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:15.665675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:15.666026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:15.666735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:15.683224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:15.714206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1168/2370] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:15 2022] POST /identity/v3/projects => generated 354 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:15.718626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:15.732883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac245b-ac77-43ea-8101-944f748625b7 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:15.802730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd770292b4041bb9bb9363f989bf9c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce3bc99858e54ca3a7846caa09182bc2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:15.803356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:15.803762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:15.804128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:15.804987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:15.805367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:15.807453 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:15.823647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1203/2371] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:15.868961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:15.880860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47395b6b-5dd7-4b51-b521-0a45a0a63827 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:15.885596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfd770292b4041bb9bb9363f989bf9c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce3bc99858e54ca3a7846caa09182bc2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:15.886230 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:15.886514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:15.886789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] PATH_INFO: `/v3/projects/da75f1b8c9524775820236f15dfff8cd` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:15.888002 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorizing `identity:delete_project(project_id=da75f1b8c9524775820236f15dfff8cd)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:15.890792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:15.919139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:15.963680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da75f1b8c9524775820236f15dfff8cd', 'request_id': 'req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:15.969616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da75f1b8c9524775820236f15dfff8cd', 'request_id': 'req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:15.977227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da75f1b8c9524775820236f15dfff8cd', 'request_id': 'req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:15.994178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] The token cache is being invalidate because project da75f1b8c9524775820236f15dfff8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:15.994464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:15.995112 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-31c10684-f901-4d18-b65c-d2cc8d2a7ebf tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:15.997060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1169/2372] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:15 2022] DELETE /identity/v3/projects/da75f1b8c9524775820236f15dfff8cd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:49:16.221603 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:16.225405 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:16.298980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b66359-0dc8-4015-b457-794916f71099 tempest-VolumeQuotaClassesTest-1543444888 tempest-VolumeQuotaClassesTest-1543444888-project-admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:16.353720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:16.372137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:16.372915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:16.373314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:16.373769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] PATH_INFO: `/v3/users/b0065f5c10bf462999dd554f182775a3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:16.375111 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0065f5c10bf462999dd554f182775a3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:16.377163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:16.442494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0065f5c10bf462999dd554f182775a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:16.443638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0065f5c10bf462999dd554f182775a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:16.451213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0065f5c10bf462999dd554f182775a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:16.452004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0065f5c10bf462999dd554f182775a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:16.462779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0065f5c10bf462999dd554f182775a3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:16.484151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1204/2373] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:16 2022] DELETE /identity/v3/users/b0065f5c10bf462999dd554f182775a3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:16.489705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:16.491169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:16.491849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:16.512962 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:16.514545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:16.592410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1170/2374] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:49:16.602280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:16.603880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:16.675925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cf1d4-af10-4cbd-aba9-938efbfdf2a1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:16.682874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:16.683536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:16.683826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:16.684103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:16.698226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:16.700075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:16.713098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1205/2375] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:17.132443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:17.133913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:17.147750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ceb82d-b832-4cb9-9543-c28bf944945b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:17.152523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:17.153185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:17.153474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:17.153755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] PATH_INFO: `/v3/projects/7a89a9ded1fd41b385fc1ce46d74893d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:17.154947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a89a9ded1fd41b385fc1ce46d74893d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:17.157092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:17.236525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a89a9ded1fd41b385fc1ce46d74893d', 'request_id': 'req-48fac806-7539-43b4-b354-44c0dc38a3a2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.244120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a89a9ded1fd41b385fc1ce46d74893d', 'request_id': 'req-48fac806-7539-43b4-b354-44c0dc38a3a2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.252481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a89a9ded1fd41b385fc1ce46d74893d', 'request_id': 'req-48fac806-7539-43b4-b354-44c0dc38a3a2'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.269211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] The token cache is being invalidate because project 7a89a9ded1fd41b385fc1ce46d74893d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:17.269496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.270154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.272343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1171/2376] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:17 2022] DELETE /identity/v3/projects/7a89a9ded1fd41b385fc1ce46d74893d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:17.281910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:17.294794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad56050-d26d-45e8-a564-9c2d9fc98288 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:17.362711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:17.363799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:17.364298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:17.364629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] PATH_INFO: `/v3/users/dfd770292b4041bb9bb9363f989bf9c8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:17.366041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd770292b4041bb9bb9363f989bf9c8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:17.368385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:17.374356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:17.429231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd770292b4041bb9bb9363f989bf9c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.429910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd770292b4041bb9bb9363f989bf9c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.441269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd770292b4041bb9bb9363f989bf9c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.442272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd770292b4041bb9bb9363f989bf9c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.454290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-10e1a8a4-e073-473a-b68b-adc30a80cc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd770292b4041bb9bb9363f989bf9c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.474783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1206/2377] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:17 2022] DELETE /identity/v3/users/dfd770292b4041bb9bb9363f989bf9c8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:17.791140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:17.792713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:17.816958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fac806-7539-43b4-b354-44c0dc38a3a2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:17.857064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:17.891058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:17.891732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:17.892066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:17.894264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] PATH_INFO: `/v3/projects/ce3bc99858e54ca3a7846caa09182bc2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:17.894883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce3bc99858e54ca3a7846caa09182bc2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:17.896970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:17.973171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce3bc99858e54ca3a7846caa09182bc2', 'request_id': 'req-834c4fe9-3640-4a58-8977-af580d5c8c51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.980050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce3bc99858e54ca3a7846caa09182bc2', 'request_id': 'req-834c4fe9-3640-4a58-8977-af580d5c8c51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:17.992645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce3bc99858e54ca3a7846caa09182bc2', 'request_id': 'req-834c4fe9-3640-4a58-8977-af580d5c8c51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:18.010535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] The token cache is being invalidate because project ce3bc99858e54ca3a7846caa09182bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:18.010934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:18.011615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-834c4fe9-3640-4a58-8977-af580d5c8c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:18.013571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1172/2378] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:17 2022] DELETE /identity/v3/projects/ce3bc99858e54ca3a7846caa09182bc2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:18.019220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ba35-de96-4f2d-b1df-47383e3dd187 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.019606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ba35-de96-4f2d-b1df-47383e3dd187 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.019962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178ba35-de96-4f2d-b1df-47383e3dd187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.045854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1178ba35-de96-4f2d-b1df-47383e3dd187 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:18.047348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1178ba35-de96-4f2d-b1df-47383e3dd187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:18.135336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1207/2379] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:18.137394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a281b-a3c3-4b85-8343-6de695b9b676 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.138043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a281b-a3c3-4b85-8343-6de695b9b676 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.138500 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a281b-a3c3-4b85-8343-6de695b9b676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.160994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-257a281b-a3c3-4b85-8343-6de695b9b676 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:18.162379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-257a281b-a3c3-4b85-8343-6de695b9b676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:18.172259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1173/2380] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:49:18.177188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178ba35-de96-4f2d-b1df-47383e3dd187 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.193598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178ba35-de96-4f2d-b1df-47383e3dd187 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.200130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.200757 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.201038 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.201313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.201977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:18.202294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.204284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.226758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1208/2381] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:18.228806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a281b-a3c3-4b85-8343-6de695b9b676 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.239001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a281b-a3c3-4b85-8343-6de695b9b676 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.243687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.244715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.245015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.245687 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.247326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.250917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.283116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1174/2382] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:18 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:18.287667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.298830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacda32-3f16-476f-96fd-00cbfa5a9463 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.369488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.370122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.370238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.370890 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.371400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.373411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.392456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:18.402604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1209/2383] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:18 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:18.404057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.416331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cb3ab-9037-4261-a56b-00b25d1f9722 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.422591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.423433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.423776 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.424115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.424834 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.426505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.438297 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1175/2384] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:18.442544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.455123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d3200-f82e-46df-8a26-121ee08b4b74 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.459966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cece89f-13ed-4e12-80bc-9ae5fc19e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.460624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece89f-13ed-4e12-80bc-9ae5fc19e5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.460970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece89f-13ed-4e12-80bc-9ae5fc19e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.461307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cece89f-13ed-4e12-80bc-9ae5fc19e5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.462025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cece89f-13ed-4e12-80bc-9ae5fc19e5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.463898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cece89f-13ed-4e12-80bc-9ae5fc19e5c2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.480559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1210/2385] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:18.481711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.492597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f86a0-98d5-4bdf-a010-1768986db35d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.497413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.498076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.498376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.498648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] PATH_INFO: `/v3/projects/3f39cfc0235142458dc3e2423b03ecc8/users/7439ae11914d401799358ef24cf58a25/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.514877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f39cfc0235142458dc3e2423b03ecc8, user_id=7439ae11914d401799358ef24cf58a25, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.518532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.536018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1176/2386] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:18 2022] PUT /identity/v3/projects/3f39cfc0235142458dc3e2423b03ecc8/users/7439ae11914d401799358ef24cf58a25/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:18.548422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9698e7c-3b57-4466-a7bb-982bb14921cc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.548799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9698e7c-3b57-4466-a7bb-982bb14921cc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.549087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9698e7c-3b57-4466-a7bb-982bb14921cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.592790 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f9698e7c-3b57-4466-a7bb-982bb14921cc None None] MFA Rules not processed for user `7439ae11914d401799358ef24cf58a25`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:18.594561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f9698e7c-3b57-4466-a7bb-982bb14921cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:18.676069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1211/2387] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:49:18.677385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.760639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:18.765060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb5b2c-e648-4ae4-be1a-1493fb138439 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.777438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.778061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.778468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.778747 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.779416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.782235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.817594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1177/2388] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:18 2022] POST /identity/v3/projects => generated 399 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:18.821391 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9698e7c-3b57-4466-a7bb-982bb14921cc None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.833408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9698e7c-3b57-4466-a7bb-982bb14921cc None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.903157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.903804 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.904124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.904390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.905114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.907765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.936372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1212/2389] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:18 2022] POST /identity/v3/users => generated 306 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:49:18.937719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.949672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42a16-210d-4a7f-b16f-8e2dc018eec8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.956172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.956789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.957072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.957367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:18.958001 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:18.959667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:18.974778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1178/2390] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:18.976012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:18.987738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d52862-6d8b-4d63-a9b8-34bd353bf8a6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:18.993904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:18.994585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:18.994861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:18.995138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] PATH_INFO: `/v3/projects/18a23ad3c36d42589225abc34131661f/users/d44290662ac94759bd66009724f032d8/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:19.014707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a23ad3c36d42589225abc34131661f, user_id=d44290662ac94759bd66009724f032d8, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:19.017659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:19.039161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1213/2391] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:18 2022] PUT /identity/v3/projects/18a23ad3c36d42589225abc34131661f/users/d44290662ac94759bd66009724f032d8/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:19.040737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:19.052724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d0df-29ab-4911-b6fa-9388504c95f7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:19.116378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a51d76-347f-4c27-8bbb-315d082740ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:19.116999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a51d76-347f-4c27-8bbb-315d082740ee admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:19.117278 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a51d76-347f-4c27-8bbb-315d082740ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:19.117549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a51d76-347f-4c27-8bbb-315d082740ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:19.118205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a51d76-347f-4c27-8bbb-315d082740ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:19.119885 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a51d76-347f-4c27-8bbb-315d082740ee admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:19.133552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1179/2392] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:19.135051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:19.147513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9451-c9fb-4e3a-97e1-7f6093b6d6ce admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:19.153249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:19.154220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:19.154686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:19.155330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] PATH_INFO: `/v3/projects/18a23ad3c36d42589225abc34131661f/users/d44290662ac94759bd66009724f032d8/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:19.157662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a23ad3c36d42589225abc34131661f, user_id=d44290662ac94759bd66009724f032d8, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:19.160511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:19.174954 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1214/2393] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:19 2022] PUT /identity/v3/projects/18a23ad3c36d42589225abc34131661f/users/d44290662ac94759bd66009724f032d8/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:19.185754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134bea-7970-4c66-abc7-09e12efe8955 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:19.186048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134bea-7970-4c66-abc7-09e12efe8955 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:19.186312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134bea-7970-4c66-abc7-09e12efe8955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:19.230362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-25134bea-7970-4c66-abc7-09e12efe8955 None None] MFA Rules not processed for user `d44290662ac94759bd66009724f032d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:19.231974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-25134bea-7970-4c66-abc7-09e12efe8955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:19.353576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1180/2394] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:19 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:19.354780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:19.356612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:19.395305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:19.424887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e1c59-40a7-4314-b931-052ab4b5ff06 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:19.429039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:19.429911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:19.430182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:19.430462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:19.443956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:19.445470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:19.456331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1215/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:19 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:49:19.553346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134bea-7970-4c66-abc7-09e12efe8955 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:19.564176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134bea-7970-4c66-abc7-09e12efe8955 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:19.567780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf6b4d-bdcc-434b-878a-f9df15b6af68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:19.568613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6b4d-bdcc-434b-878a-f9df15b6af68 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:19.568895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6b4d-bdcc-434b-878a-f9df15b6af68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:19.569176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6b4d-bdcc-434b-878a-f9df15b6af68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:19.636783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:19.649814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf6b4d-bdcc-434b-878a-f9df15b6af68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:19.651310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf6b4d-bdcc-434b-878a-f9df15b6af68 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:19.662013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1181/2396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:23.917456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:23.986497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5c98-1ffc-4923-a834-074955a1298e service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:23.992257 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:23.992905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:23.993179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:23.993754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] PATH_INFO: `/v3/users/7439ae11914d401799358ef24cf58a25` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:23.995041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7439ae11914d401799358ef24cf58a25)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:23.997032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:24.046332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7439ae11914d401799358ef24cf58a25'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.046998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7439ae11914d401799358ef24cf58a25'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.054239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7439ae11914d401799358ef24cf58a25'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.054928 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7439ae11914d401799358ef24cf58a25'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.063814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7439ae11914d401799358ef24cf58a25'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.082944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1216/2397] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:23 2022] DELETE /identity/v3/users/7439ae11914d401799358ef24cf58a25 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:24.087557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:24.088066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:24.088165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:24.110562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:24.112930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:24.174761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1182/2398] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:24.184837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:24.186710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:24.252176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d178d3-8c05-4609-945b-33f587a8496d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:24.256786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:24.257628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:24.257832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:24.258213 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:24.267203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:24.270439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:24.273690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:24.286360 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1217/2399] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:24.592068 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:24.593564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:24.607220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1be65a-4f4b-4acc-aa5a-b39f5eb63688 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:24.612704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:24.614404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:24.614404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:24.614404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] PATH_INFO: `/v3/projects/3f39cfc0235142458dc3e2423b03ecc8` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:24.614978 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f39cfc0235142458dc3e2423b03ecc8)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:24.617046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:24.674926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f39cfc0235142458dc3e2423b03ecc8', 'request_id': 'req-c47b5b24-8949-4591-ab2f-9e5c6884779d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.682115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f39cfc0235142458dc3e2423b03ecc8', 'request_id': 'req-c47b5b24-8949-4591-ab2f-9e5c6884779d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.689782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f39cfc0235142458dc3e2423b03ecc8', 'request_id': 'req-c47b5b24-8949-4591-ab2f-9e5c6884779d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.703759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] The token cache is being invalidate because project 3f39cfc0235142458dc3e2423b03ecc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:24.704199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.705469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.710381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1183/2400] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:24 2022] DELETE /identity/v3/projects/3f39cfc0235142458dc3e2423b03ecc8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:24.711462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:24.722481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44a383-8d98-46a0-8ed6-379766950e5f service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:24.786057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:24.786731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:24.787023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:24.787354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] PATH_INFO: `/v3/users/d44290662ac94759bd66009724f032d8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:24.788460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44290662ac94759bd66009724f032d8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:24.790444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:24.845085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44290662ac94759bd66009724f032d8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.845860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44290662ac94759bd66009724f032d8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.853121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44290662ac94759bd66009724f032d8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.853885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44290662ac94759bd66009724f032d8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.862954 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-840c0570-01db-4261-bf6b-5ccab9050876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44290662ac94759bd66009724f032d8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:24.879983 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1218/2401] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:24 2022] DELETE /identity/v3/users/d44290662ac94759bd66009724f032d8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:25.166115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.168050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:25.190082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b5b24-8949-4591-ab2f-9e5c6884779d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:25.258466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:25.259111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.259488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.259770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] PATH_INFO: `/v3/projects/18a23ad3c36d42589225abc34131661f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.260985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a23ad3c36d42589225abc34131661f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:25.262906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:25.280671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:25.320154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a23ad3c36d42589225abc34131661f', 'request_id': 'req-9a4ee3dc-d618-4703-bacc-e0ba16162c58'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:25.327543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a23ad3c36d42589225abc34131661f', 'request_id': 'req-9a4ee3dc-d618-4703-bacc-e0ba16162c58'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:25.339662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a23ad3c36d42589225abc34131661f', 'request_id': 'req-9a4ee3dc-d618-4703-bacc-e0ba16162c58'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:25.355251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] The token cache is being invalidate because project 18a23ad3c36d42589225abc34131661f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:25.355450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:25.356062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9a4ee3dc-d618-4703-bacc-e0ba16162c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:25.357615 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1184/2402] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:25 2022] DELETE /identity/v3/projects/18a23ad3c36d42589225abc34131661f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:25.363273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8deec4-1af4-463d-84e6-80dfd0b3db25 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.363768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8deec4-1af4-463d-84e6-80dfd0b3db25 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.363871 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8deec4-1af4-463d-84e6-80dfd0b3db25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.388926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-cb8deec4-1af4-463d-84e6-80dfd0b3db25 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:25.390219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cb8deec4-1af4-463d-84e6-80dfd0b3db25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:25.467244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1219/2403] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:25.474967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d2342-8915-4a76-af5c-f93550328f84 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.475302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d2342-8915-4a76-af5c-f93550328f84 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.475583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d2342-8915-4a76-af5c-f93550328f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.497514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0b9d2342-8915-4a76-af5c-f93550328f84 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:25.499561 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0b9d2342-8915-4a76-af5c-f93550328f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:25.508818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1185/2404] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:25.513244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8deec4-1af4-463d-84e6-80dfd0b3db25 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.524127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8deec4-1af4-463d-84e6-80dfd0b3db25 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:25.531237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:25.531829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.532115 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.532393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.533054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:25.533361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:25.535100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:25.550542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1220/2405] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:25.554956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d2342-8915-4a76-af5c-f93550328f84 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.565671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d2342-8915-4a76-af5c-f93550328f84 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:25.571628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:25.572240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.572537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.572832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.573475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:25.575576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:25.612749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1186/2406] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:25 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:25.616912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.623699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:25.633004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a45ac-0410-4322-bf83-601a75f66be3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:25.700095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:25.700717 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.700998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.701279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.701896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:25.704215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:25.727680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1221/2407] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:25 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:25.731930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.754638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc52b5-d3bd-4c17-b797-aa83c99d5ecf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:25.759710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:25.760326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.760637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.760918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.761645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:25.763891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:25.775719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1187/2408] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:49:25.779845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.790675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412d16d-893e-406c-868f-537c5d8a8824 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:25.796324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1933bc-96b4-4d6c-b4d6-960615e849df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:25.796938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1933bc-96b4-4d6c-b4d6-960615e849df admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.797243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1933bc-96b4-4d6c-b4d6-960615e849df admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.797503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1933bc-96b4-4d6c-b4d6-960615e849df admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.798264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1933bc-96b4-4d6c-b4d6-960615e849df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:25.799930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1933bc-96b4-4d6c-b4d6-960615e849df admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:25.812271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1222/2409] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:25.816636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.827157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b69e-91e8-4cde-b206-e03c7257e071 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:25.832252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:25.832911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.833195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.833472 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] PATH_INFO: `/v3/projects/abd7f34b040044d393cd6a8a6ff40d59/users/daa4db73ad724f7382ba408b33067c0c/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.849680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd7f34b040044d393cd6a8a6ff40d59, user_id=daa4db73ad724f7382ba408b33067c0c, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:25.852400 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:25.867229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1188/2410] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:25 2022] PUT /identity/v3/projects/abd7f34b040044d393cd6a8a6ff40d59/users/daa4db73ad724f7382ba408b33067c0c/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:25.878793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e52b32-4f37-4b12-b37f-1b525dd8b242 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:25.879081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e52b32-4f37-4b12-b37f-1b525dd8b242 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:25.879423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e52b32-4f37-4b12-b37f-1b525dd8b242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:25.909011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-56e52b32-4f37-4b12-b37f-1b525dd8b242 None None] MFA Rules not processed for user `daa4db73ad724f7382ba408b33067c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:25.910276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-56e52b32-4f37-4b12-b37f-1b525dd8b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:25.979211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1223/2411] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:25.991480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:25.994284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:26.059833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67cd38-1d96-4e5e-84fd-b4b2ed3e06df admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:26.067652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc111627-f4e3-4182-8b2b-77386efb2dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:26.068516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc111627-f4e3-4182-8b2b-77386efb2dba service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:26.068883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc111627-f4e3-4182-8b2b-77386efb2dba service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:26.069247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc111627-f4e3-4182-8b2b-77386efb2dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:26.080479 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc111627-f4e3-4182-8b2b-77386efb2dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:26.082288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc111627-f4e3-4182-8b2b-77386efb2dba service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:26.097253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1189/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:26.123628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e52b32-4f37-4b12-b37f-1b525dd8b242 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:26.191578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e52b32-4f37-4b12-b37f-1b525dd8b242 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:26.196846 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:26.197504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:26.197783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:26.198056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] PATH_INFO: `/v3/users/daa4db73ad724f7382ba408b33067c0c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:26.199355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa4db73ad724f7382ba408b33067c0c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:26.201299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:26.222440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:26.261839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa4db73ad724f7382ba408b33067c0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:26.262683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa4db73ad724f7382ba408b33067c0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:26.271147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa4db73ad724f7382ba408b33067c0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:26.271842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa4db73ad724f7382ba408b33067c0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:26.280399 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa4db73ad724f7382ba408b33067c0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:26.303879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1224/2413] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:26 2022] DELETE /identity/v3/users/daa4db73ad724f7382ba408b33067c0c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:26.308628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:26.309454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:26.309454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:26.331417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:26.333007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:26.377487 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:26.404651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1190/2414] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:49:26.420800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:26.422555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:26.492807 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d437448-57d9-471d-957f-476eea223510 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:26.496604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f636b-ffad-4027-b969-adb323c719dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:26.497217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f636b-ffad-4027-b969-adb323c719dd service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:26.497499 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f636b-ffad-4027-b969-adb323c719dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:26.497771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f636b-ffad-4027-b969-adb323c719dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:26.507370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f636b-ffad-4027-b969-adb323c719dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:26.508819 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f636b-ffad-4027-b969-adb323c719dd service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:26.518532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1225/2415] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:27.072998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.075558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:27.097350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b08b5-b74b-4ad6-8600-1d8d6a64d420 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.102291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.102939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.103246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.103529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] PATH_INFO: `/v3/projects/abd7f34b040044d393cd6a8a6ff40d59` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.104661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd7f34b040044d393cd6a8a6ff40d59)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.106579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.156475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd7f34b040044d393cd6a8a6ff40d59', 'request_id': 'req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:27.162119 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd7f34b040044d393cd6a8a6ff40d59', 'request_id': 'req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:27.168409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd7f34b040044d393cd6a8a6ff40d59', 'request_id': 'req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:27.182244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] The token cache is being invalidate because project abd7f34b040044d393cd6a8a6ff40d59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:27.182536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:27.183179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9e508ca7-75a0-4fc6-a3fa-dd9376ff1ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:27.184724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1191/2416] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:27 2022] DELETE /identity/v3/projects/abd7f34b040044d393cd6a8a6ff40d59 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:27.190165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2a74e-26d7-4565-baed-98b4f6031805 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.190457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2a74e-26d7-4565-baed-98b4f6031805 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.190736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2a74e-26d7-4565-baed-98b4f6031805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.210707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a2d2a74e-26d7-4565-baed-98b4f6031805 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:27.211969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a2d2a74e-26d7-4565-baed-98b4f6031805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:27.273864 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1226/2417] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:27.281810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fde965-0d95-443a-b370-4b184f6bd3bd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.282300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fde965-0d95-443a-b370-4b184f6bd3bd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.282710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fde965-0d95-443a-b370-4b184f6bd3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.304762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-41fde965-0d95-443a-b370-4b184f6bd3bd None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:27.305967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-41fde965-0d95-443a-b370-4b184f6bd3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:27.315272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1192/2418] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:27.319760 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2a74e-26d7-4565-baed-98b4f6031805 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.333459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2a74e-26d7-4565-baed-98b4f6031805 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.342587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.343667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.344171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.344641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.345537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:27.345847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.347537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.360699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1227/2419] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:27.365249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fde965-0d95-443a-b370-4b184f6bd3bd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.374775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fde965-0d95-443a-b370-4b184f6bd3bd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.381536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.382151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.382429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.382698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.383385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.385383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.415070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1193/2420] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:27 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:49:27.419306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.430608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f3cc-2334-46e7-9977-78c17ab4e010 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.450669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:27.498058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.498736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.499027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.499339 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.499981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.502042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.529556 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1228/2421] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:27 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:49:27.531580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.543674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8bf57-64e8-4da4-b429-c8470b0dcd71 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.548775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.549515 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.549907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.550277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.551033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.553222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.565631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1194/2422] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:27.569763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.580616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d99bb-9df8-4280-8440-f96a539caba6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.585793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44356ba8-31fd-456e-a831-3da9c0dc7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.586718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356ba8-31fd-456e-a831-3da9c0dc7fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.586922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356ba8-31fd-456e-a831-3da9c0dc7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.587290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356ba8-31fd-456e-a831-3da9c0dc7fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.587953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44356ba8-31fd-456e-a831-3da9c0dc7fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.589638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44356ba8-31fd-456e-a831-3da9c0dc7fb9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.602515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1229/2423] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:27.607263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.618900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d694ec5-99bf-42b2-9994-eb19be26a61e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.626661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.626979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.627179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.627488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] PATH_INFO: `/v3/projects/3f8763ee5618496e8ba351c5efe18f6a/users/9c8b5e70a20340c7b6d3dd974578ed44/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.645413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8763ee5618496e8ba351c5efe18f6a, user_id=9c8b5e70a20340c7b6d3dd974578ed44, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.648412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.665385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1195/2424] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:27 2022] PUT /identity/v3/projects/3f8763ee5618496e8ba351c5efe18f6a/users/9c8b5e70a20340c7b6d3dd974578ed44/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:27.675754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc89c62-8886-452c-b18d-c097e9d9fe5c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.676055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc89c62-8886-452c-b18d-c097e9d9fe5c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.676318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc89c62-8886-452c-b18d-c097e9d9fe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.720740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9dc89c62-8886-452c-b18d-c097e9d9fe5c None None] MFA Rules not processed for user `9c8b5e70a20340c7b6d3dd974578ed44`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:27.722034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9dc89c62-8886-452c-b18d-c097e9d9fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:27.792440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1230/2425] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:27.804571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:27.806535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:27.866314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c481ed-e010-4016-b929-01e98a8d44f5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:27.870116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:27.870732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.871016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.871331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.880034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:27.881504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:27.890606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1196/2426] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:28.149341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46258d97-1666-49f2-bbfa-819da77642a2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:28.149640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46258d97-1666-49f2-bbfa-819da77642a2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:28.149925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46258d97-1666-49f2-bbfa-819da77642a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:28.171904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-46258d97-1666-49f2-bbfa-819da77642a2 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:28.173198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-46258d97-1666-49f2-bbfa-819da77642a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:28.242611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1231/2427] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:49:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:28.255601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:28.257034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:28.268478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:28.333041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d793126-193f-48d6-bd04-612f041c1ab4 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:28.339311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:28.339923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:28.340253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:28.340525 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:28.349263 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:28.350839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:28.358777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1197/2428] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:49:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:49:38.235731 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:38.355641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=29 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:47.734482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186fc99-fd26-4f85-b131-4a0b438c4d2d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:47.735415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186fc99-fd26-4f85-b131-4a0b438c4d2d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:47.735848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186fc99-fd26-4f85-b131-4a0b438c4d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:47.758143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a186fc99-fd26-4f85-b131-4a0b438c4d2d None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:47.759336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a186fc99-fd26-4f85-b131-4a0b438c4d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:47.767987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1232/2429] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 06:49:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:47.782509 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:47.793958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e62adc-8738-455e-9ff9-803d639910b5 service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:47.800073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5303fff-463d-43e1-b0d5-6ba7b3302907 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:47.800718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5303fff-463d-43e1-b0d5-6ba7b3302907 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:47.801021 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5303fff-463d-43e1-b0d5-6ba7b3302907 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:47.801267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5303fff-463d-43e1-b0d5-6ba7b3302907 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:47.810902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5303fff-463d-43e1-b0d5-6ba7b3302907 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:47.812526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5303fff-463d-43e1-b0d5-6ba7b3302907 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:47.821542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1198/2430] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:49:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:49:47.830054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186fc99-fd26-4f85-b131-4a0b438c4d2d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:47.946242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186fc99-fd26-4f85-b131-4a0b438c4d2d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:47.953074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:47.954101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:47.954481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:47.954894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] PATH_INFO: `/v3/users/9c8b5e70a20340c7b6d3dd974578ed44` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:47.956308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8b5e70a20340c7b6d3dd974578ed44)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:47.958429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:48.038210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8b5e70a20340c7b6d3dd974578ed44'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.038970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8b5e70a20340c7b6d3dd974578ed44'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.048956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8b5e70a20340c7b6d3dd974578ed44'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.050000 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8b5e70a20340c7b6d3dd974578ed44'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.063712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8b5e70a20340c7b6d3dd974578ed44'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.097396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1233/2431] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:47 2022] DELETE /identity/v3/users/9c8b5e70a20340c7b6d3dd974578ed44 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:49:48.102617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.102998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.103444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:48.124423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:48.125756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:48.191135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1199/2432] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:48.202305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:48.204043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:48.271249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1820bd-c9fc-44c4-97d1-7cd04e60e973 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:48.275040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cc161-4ee3-425f-8287-12ae24997d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:48.275630 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc161-4ee3-425f-8287-12ae24997d09 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.275942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc161-4ee3-425f-8287-12ae24997d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.276216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc161-4ee3-425f-8287-12ae24997d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:48.286101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cc161-4ee3-425f-8287-12ae24997d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:48.287590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cc161-4ee3-425f-8287-12ae24997d09 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:48.297244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1234/2433] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:48.573737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:48.575409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:48.588807 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2d21f-adc0-4ea0-99b3-bed51e4ebdbe None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:48.593567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:48.594204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.594508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.594786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] PATH_INFO: `/v3/projects/3f8763ee5618496e8ba351c5efe18f6a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:48.595988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8763ee5618496e8ba351c5efe18f6a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:48.597915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:48.647478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8763ee5618496e8ba351c5efe18f6a', 'request_id': 'req-5201fa17-5900-4fb4-8040-c1e33fd598fe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.654054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8763ee5618496e8ba351c5efe18f6a', 'request_id': 'req-5201fa17-5900-4fb4-8040-c1e33fd598fe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.660659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8763ee5618496e8ba351c5efe18f6a', 'request_id': 'req-5201fa17-5900-4fb4-8040-c1e33fd598fe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.675092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] The token cache is being invalidate because project 3f8763ee5618496e8ba351c5efe18f6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:48.675427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.676138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5201fa17-5900-4fb4-8040-c1e33fd598fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:48.677637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1200/2434] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:48 2022] DELETE /identity/v3/projects/3f8763ee5618496e8ba351c5efe18f6a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:48.684267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958f074-e43d-48be-bb87-805c803f3dae None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.684557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958f074-e43d-48be-bb87-805c803f3dae None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.684834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958f074-e43d-48be-bb87-805c803f3dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:48.708907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e958f074-e43d-48be-bb87-805c803f3dae None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:48.710142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e958f074-e43d-48be-bb87-805c803f3dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:48.721223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:48.774325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1235/2435] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:48.782454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33070-f577-4004-84da-8a5701955d8d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.782743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33070-f577-4004-84da-8a5701955d8d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.783307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d33070-f577-4004-84da-8a5701955d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:48.803000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c6d33070-f577-4004-84da-8a5701955d8d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:48.804340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c6d33070-f577-4004-84da-8a5701955d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:48.813262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1201/2436] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:48.819939 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958f074-e43d-48be-bb87-805c803f3dae None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:48.833938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958f074-e43d-48be-bb87-805c803f3dae None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:48.839624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:48.840233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.840515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.840781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:48.841428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:48.841763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:48.843550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:48.855897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1236/2437] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:48.860387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33070-f577-4004-84da-8a5701955d8d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:48.873831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d33070-f577-4004-84da-8a5701955d8d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:48.882308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:48.883123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.883439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.883696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:48.884332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:48.886251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:48.918104 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1202/2438] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:48 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:49:48.922784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:48.933959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52267b89-03b7-4e85-8899-4db561f0647f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:48.998022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:48.998908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:48.999412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:48.999788 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:49.000689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:49.003682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:49.032410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1237/2439] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:48 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:49.033576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:49.048550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72777a98-be72-4100-8d9f-7db65c8ec6f6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:49.053731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:49.054339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:49.054617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:49.054892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:49.055581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:49.057195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:49.067838 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1203/2440] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:49.072132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:49.082512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5a75-a3b9-4bf9-9746-484abadc236b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:49.087853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278af63-55d5-4ef8-b767-065f89deef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:49.088459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278af63-55d5-4ef8-b767-065f89deef44 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:49.088744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278af63-55d5-4ef8-b767-065f89deef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:49.089019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278af63-55d5-4ef8-b767-065f89deef44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:49.089702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278af63-55d5-4ef8-b767-065f89deef44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:49.091404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278af63-55d5-4ef8-b767-065f89deef44 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:49.106632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1238/2441] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:49.107906 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:49.118315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712c75b-5e50-43b3-ada7-0c7307fadd34 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:49.123073 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:49.123759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:49.124041 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:49.124314 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] PATH_INFO: `/v3/projects/481a2aeb141e48bb8e94ec53a53b5396/users/579a71fafffe46bba09a1e1bbf074d5e/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:49.140914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] RBAC: Authorizing `identity:create_grant(project_id=481a2aeb141e48bb8e94ec53a53b5396, user_id=579a71fafffe46bba09a1e1bbf074d5e, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:49.143661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:49.158130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1204/2442] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:49 2022] PUT /identity/v3/projects/481a2aeb141e48bb8e94ec53a53b5396/users/579a71fafffe46bba09a1e1bbf074d5e/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:49.168858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d1ab3-0788-476e-ab8a-3a384b0780e8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:49.169151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d1ab3-0788-476e-ab8a-3a384b0780e8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:49.169427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d1ab3-0788-476e-ab8a-3a384b0780e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:49.200255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5d1d1ab3-0788-476e-ab8a-3a384b0780e8 None None] MFA Rules not processed for user `579a71fafffe46bba09a1e1bbf074d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:49.201488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5d1d1ab3-0788-476e-ab8a-3a384b0780e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:49.272910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1239/2443] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:49.283767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:49.285689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:49.296252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:49.346259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcccea-8332-4711-a370-7329d9afc50d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:49.350740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa0a4-4d08-4d70-930e-e77d03e97b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:49.351676 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa0a4-4d08-4d70-930e-e77d03e97b76 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:49.352033 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa0a4-4d08-4d70-930e-e77d03e97b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:49.352309 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa0a4-4d08-4d70-930e-e77d03e97b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:49.361503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fa0a4-4d08-4d70-930e-e77d03e97b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:49.362944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fa0a4-4d08-4d70-930e-e77d03e97b76 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:49.372070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1205/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:50.781936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:50.782500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:50.782500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:50.803611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] MFA Rules not processed for user `579a71fafffe46bba09a1e1bbf074d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:50.804815 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:50.813383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1240/2445] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:49:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:59.368383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=23 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:00.797337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:08.182910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa96ae4-1ee6-4948-9117-d0250c4df87b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:08.183700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa96ae4-1ee6-4948-9117-d0250c4df87b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:08.183700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa96ae4-1ee6-4948-9117-d0250c4df87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:08.208322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-efa96ae4-1ee6-4948-9117-d0250c4df87b None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:08.209476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-efa96ae4-1ee6-4948-9117-d0250c4df87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:08.279145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1206/2446] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:50:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:08.291176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:08.292627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:08.365464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3091b7-95c9-4161-a7ef-a2b2bbca93f0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:08.372762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:08.373423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:08.373714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:08.373991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:08.384133 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:08.385750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:08.394256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1241/2447] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:50:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:50:08.993210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:08.993504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:08.993782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:09.014224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:09.015483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:09.023288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1207/2448] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 06:50:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:09.036266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:09.049147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28872375-5b18-4c04-a713-d81fa84b6555 service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:09.055090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b09386-b03a-42d9-94f9-1e51be5353e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:09.055733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b09386-b03a-42d9-94f9-1e51be5353e3 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:09.056016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b09386-b03a-42d9-94f9-1e51be5353e3 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:09.056290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b09386-b03a-42d9-94f9-1e51be5353e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:09.070984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b09386-b03a-42d9-94f9-1e51be5353e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:09.072833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b09386-b03a-42d9-94f9-1e51be5353e3 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:09.083371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1242/2449] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:50:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:50:09.119045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:09.120545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:09.196228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452368b-08c7-4d54-94c3-56bcf385cdc3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:09.201000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:09.201701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:09.202058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:09.202404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:09.204275 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:09.278960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:09.280629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:09.291151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1208/2450] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:09.332424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9218-b9fa-4f4b-9ecc-6f01f54bd961 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:09.332959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9218-b9fa-4f4b-9ecc-6f01f54bd961 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:09.332959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9218-b9fa-4f4b-9ecc-6f01f54bd961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:09.352182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-acbd9218-b9fa-4f4b-9ecc-6f01f54bd961 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:09.353390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-acbd9218-b9fa-4f4b-9ecc-6f01f54bd961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:09.360736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1243/2451] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:50:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:09.373031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:09.385298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea757f65-f0b6-4296-88d3-1d8a672a66fd service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:09.391959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:09.393308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:09.393308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:09.393308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:09.402671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:09.404356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:09.413175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1209/2452] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:50:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:50:09.779712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3375-f575-46ec-8c2b-33a38aeaeacf None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:09.780501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3375-f575-46ec-8c2b-33a38aeaeacf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:09.781052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3375-f575-46ec-8c2b-33a38aeaeacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:09.804509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-727f3375-f575-46ec-8c2b-33a38aeaeacf None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:09.805754 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-727f3375-f575-46ec-8c2b-33a38aeaeacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:09.814448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1244/2453] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 06:50:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:09.827439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:09.839242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5bff1-9eee-4d46-b45c-ac2bc616795b service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:09.844948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641f0e4-e6bf-40b2-9578-85558ade2eea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:09.845556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641f0e4-e6bf-40b2-9578-85558ade2eea service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:09.845832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641f0e4-e6bf-40b2-9578-85558ade2eea service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:09.846102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641f0e4-e6bf-40b2-9578-85558ade2eea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:09.856598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641f0e4-e6bf-40b2-9578-85558ade2eea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:09.858194 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641f0e4-e6bf-40b2-9578-85558ade2eea service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:09.867315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1210/2454] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:50:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:50:12.081276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f3375-f575-46ec-8c2b-33a38aeaeacf None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:12.095745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f3375-f575-46ec-8c2b-33a38aeaeacf None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:12.101535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:12.102319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:12.102611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:12.102894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] PATH_INFO: `/v3/users/579a71fafffe46bba09a1e1bbf074d5e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:12.104135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=579a71fafffe46bba09a1e1bbf074d5e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:12.106508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:12.163477 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '579a71fafffe46bba09a1e1bbf074d5e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:12.164249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '579a71fafffe46bba09a1e1bbf074d5e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:12.172157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '579a71fafffe46bba09a1e1bbf074d5e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:12.172881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '579a71fafffe46bba09a1e1bbf074d5e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:12.182201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '579a71fafffe46bba09a1e1bbf074d5e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:12.203163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1245/2455] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:12 2022] DELETE /identity/v3/users/579a71fafffe46bba09a1e1bbf074d5e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:12.208447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:12.209145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:12.209145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:12.244103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:12.245917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:12.266238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:12.334596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1211/2456] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:12.377177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:12.379145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:12.447786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:12.471148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75393c31-1e69-45df-83af-4cce24f52bae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:12.475794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0c677-c8ec-4952-a3e2-729ac1ff3cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:12.476454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0c677-c8ec-4952-a3e2-729ac1ff3cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:12.479551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0c677-c8ec-4952-a3e2-729ac1ff3cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:12.479551 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0c677-c8ec-4952-a3e2-729ac1ff3cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:12.493348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0c677-c8ec-4952-a3e2-729ac1ff3cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:12.494919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0c677-c8ec-4952-a3e2-729ac1ff3cfb service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:12.506631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1246/2457] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:12.929988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:12.931865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:12.957669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef681d3f-752f-4cb1-884c-3453d3ea2d22 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:12.965604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:12.966486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:12.967059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:12.967765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] PATH_INFO: `/v3/projects/481a2aeb141e48bb8e94ec53a53b5396` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:12.972521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=481a2aeb141e48bb8e94ec53a53b5396)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:12.976203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.064419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481a2aeb141e48bb8e94ec53a53b5396', 'request_id': 'req-45377af4-0378-49f1-8a2a-7b8870546874'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.077480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481a2aeb141e48bb8e94ec53a53b5396', 'request_id': 'req-45377af4-0378-49f1-8a2a-7b8870546874'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.085531 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481a2aeb141e48bb8e94ec53a53b5396', 'request_id': 'req-45377af4-0378-49f1-8a2a-7b8870546874'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.103801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] The token cache is being invalidate because project 481a2aeb141e48bb8e94ec53a53b5396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:13.104255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.105102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-45377af4-0378-49f1-8a2a-7b8870546874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.110348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1212/2458] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:12 2022] DELETE /identity/v3/projects/481a2aeb141e48bb8e94ec53a53b5396 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:13.112228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fd706-f9f0-48fe-9e48-8519ad49911f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.112859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fd706-f9f0-48fe-9e48-8519ad49911f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.112859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fd706-f9f0-48fe-9e48-8519ad49911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.160629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a50fd706-f9f0-48fe-9e48-8519ad49911f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:13.163787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a50fd706-f9f0-48fe-9e48-8519ad49911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:13.253250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1247/2459] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:13.261157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f5ac-ccc6-4eaf-905c-7528de5c6b94 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.261930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f5ac-ccc6-4eaf-905c-7528de5c6b94 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.262216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910f5ac-ccc6-4eaf-905c-7528de5c6b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.300563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3910f5ac-ccc6-4eaf-905c-7528de5c6b94 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:13.301874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3910f5ac-ccc6-4eaf-905c-7528de5c6b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:13.324946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1213/2460] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:13.329378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fd706-f9f0-48fe-9e48-8519ad49911f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:13.344189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fd706-f9f0-48fe-9e48-8519ad49911f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:13.351628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:13.352677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.352905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.353228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.353946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:13.354323 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:13.356720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.378720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1248/2461] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:13.385026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910f5ac-ccc6-4eaf-905c-7528de5c6b94 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:13.396255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910f5ac-ccc6-4eaf-905c-7528de5c6b94 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:13.401103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:13.401733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.402015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.402290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.402940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:13.405167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.447071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1214/2462] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:13 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:13.451383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:13.475762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd2383-7218-469b-87aa-e4c76fe90ec1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:13.575781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:13.576617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.577044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.577956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.579238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:13.582242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.617311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:13.617811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1249/2463] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:13 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:13.622383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:13.622657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:13.637223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f442ff-6acc-44d6-87bb-4c9024f30cd6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:13.641787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424cd71-66b6-44be-9b2f-197115690f7a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:13.644956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:13.645735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.645912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.646193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.647535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:13.653590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.667415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:13.668241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.668546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.668949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] PATH_INFO: `/v3/users/1d26cf886087408097cc2e435a785454` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.670224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d26cf886087408097cc2e435a785454)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:13.672394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.674460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1250/2464] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:13.675787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:13.690851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc99836-8487-4440-b83c-17d0debbd24b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:13.699034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:13.699721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.700005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.700282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.700943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:13.703468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.730615 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1251/2465] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:13.735532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:13.753447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d26cf886087408097cc2e435a785454'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.756626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1e30e-07f0-4ed4-914e-e0140a974820 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:13.763587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed44af-2e4e-42e7-81e3-e64cd55dccaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:13.764414 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed44af-2e4e-42e7-81e3-e64cd55dccaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.764671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed44af-2e4e-42e7-81e3-e64cd55dccaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.764944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed44af-2e4e-42e7-81e3-e64cd55dccaf admin admin] PATH_INFO: `/v3/projects/cba7269cffa2467b9a1a13180d943207/users/18bc1fbb50e643918a1e5d5d1c3e8700/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.767973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d26cf886087408097cc2e435a785454'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.789435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed44af-2e4e-42e7-81e3-e64cd55dccaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba7269cffa2467b9a1a13180d943207, user_id=18bc1fbb50e643918a1e5d5d1c3e8700, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:13.789707 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d26cf886087408097cc2e435a785454'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.790394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d26cf886087408097cc2e435a785454'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.791274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d26cf886087408097cc2e435a785454'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:13.792455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed44af-2e4e-42e7-81e3-e64cd55dccaf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:13.801301 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:13.821568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1252/2466] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:13 2022] PUT /identity/v3/projects/cba7269cffa2467b9a1a13180d943207/users/18bc1fbb50e643918a1e5d5d1c3e8700/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:13.848874 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1215/2467] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:13 2022] DELETE /identity/v3/users/1d26cf886087408097cc2e435a785454 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:13.894042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:13.894441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:13.894719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:13.916750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:13.928913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] MFA Rules not processed for user `18bc1fbb50e643918a1e5d5d1c3e8700`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:13.930158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:14.046784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1253/2468] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:14.059357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.061217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:14.117711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.122131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:14.167632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569a4c-b266-40f8-867f-dc5b38b1d9ec admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.173961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c52278-ae6d-4667-9842-0bd6002d1fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.174901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c52278-ae6d-4667-9842-0bd6002d1fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.175265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c52278-ae6d-4667-9842-0bd6002d1fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.175844 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c52278-ae6d-4667-9842-0bd6002d1fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.188056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c52278-ae6d-4667-9842-0bd6002d1fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.189650 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c52278-ae6d-4667-9842-0bd6002d1fd1 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.202093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1216/2469] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:14.238182 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b12d2-bb70-4f55-87c4-31d6851ed39c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.243843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.244580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.244926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.245423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] PATH_INFO: `/v3/projects/5b4541156c1f4efab3a61dd6935638bb` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.246559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4541156c1f4efab3a61dd6935638bb)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.249205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.308970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4541156c1f4efab3a61dd6935638bb', 'request_id': 'req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:14.318328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4541156c1f4efab3a61dd6935638bb', 'request_id': 'req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:14.335022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4541156c1f4efab3a61dd6935638bb', 'request_id': 'req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:14.359887 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] The token cache is being invalidate because project 5b4541156c1f4efab3a61dd6935638bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:14.360269 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:14.361327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0d94d75b-ffc7-4a98-a1fa-939e08643b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:14.368689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1254/2470] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:14 2022] DELETE /identity/v3/projects/5b4541156c1f4efab3a61dd6935638bb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:14.373344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f197-f311-4ed6-b536-86f4d76d42a5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.374235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f197-f311-4ed6-b536-86f4d76d42a5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.375293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f197-f311-4ed6-b536-86f4d76d42a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.409016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-96d4f197-f311-4ed6-b536-86f4d76d42a5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:14.410710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-96d4f197-f311-4ed6-b536-86f4d76d42a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:14.494888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1217/2471] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:14.506695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ae05-dfef-4bda-b304-2cbac610898b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.507059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ae05-dfef-4bda-b304-2cbac610898b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.507467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ae05-dfef-4bda-b304-2cbac610898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.530509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2e87ae05-dfef-4bda-b304-2cbac610898b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:14.531850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2e87ae05-dfef-4bda-b304-2cbac610898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:14.544483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1255/2472] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:14.548113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4f197-f311-4ed6-b536-86f4d76d42a5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.566325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4f197-f311-4ed6-b536-86f4d76d42a5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.571544 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.572188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.572540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.572909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.573643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:14.573953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.575854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.588703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1218/2473] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:50:14.593221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87ae05-dfef-4bda-b304-2cbac610898b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.603239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87ae05-dfef-4bda-b304-2cbac610898b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.609008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.609618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.609876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.610147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.610780 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.612880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.638480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:14.650134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1256/2474] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:14 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:14.651418 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.663877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ef0b2-7191-436e-842d-87120e7e1886 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.736802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.737613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.737940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.738218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.738862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.740855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.766691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1219/2475] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:14 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:14.770969 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.781503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab115604-70da-4e1f-a3d2-7fa12a6da494 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.786298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.786916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.787223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.787501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.788160 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.789767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.801458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1257/2476] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:14.805558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.821319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc199f-1d92-4e16-b85e-16fb81d21e6f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.826537 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e115983-3f46-41fc-97ea-3f53f6850433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.827145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115983-3f46-41fc-97ea-3f53f6850433 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.827462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115983-3f46-41fc-97ea-3f53f6850433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.827764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115983-3f46-41fc-97ea-3f53f6850433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.828424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e115983-3f46-41fc-97ea-3f53f6850433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.830092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e115983-3f46-41fc-97ea-3f53f6850433 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.832933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:14.844737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1220/2477] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:14.849145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:14.859772 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb059e-d563-4c7f-8692-fc764694959d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:14.865779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:14.866430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.866709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.866996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] PATH_INFO: `/v3/projects/f0e866a421fc48c4b81edf38afcb5efc/users/dc822e8774a14700a31f352ee7d225c8/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.883372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e866a421fc48c4b81edf38afcb5efc, user_id=dc822e8774a14700a31f352ee7d225c8, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:14.886039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:14.902118 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1258/2478] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:14 2022] PUT /identity/v3/projects/f0e866a421fc48c4b81edf38afcb5efc/users/dc822e8774a14700a31f352ee7d225c8/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:14.913733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126c89-31e2-4c6d-84e1-0388a0a4cb74 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:14.914110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126c89-31e2-4c6d-84e1-0388a0a4cb74 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:14.914390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126c89-31e2-4c6d-84e1-0388a0a4cb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:14.945050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3a126c89-31e2-4c6d-84e1-0388a0a4cb74 None None] MFA Rules not processed for user `dc822e8774a14700a31f352ee7d225c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:14.946287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3a126c89-31e2-4c6d-84e1-0388a0a4cb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:15.019470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1221/2479] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:15.024348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.089010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a25d3-2754-41fe-b40d-a03748b0cd10 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.094429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.095037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.095349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.095629 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.096349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.098681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.130166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1259/2480] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:15 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:15.136494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a126c89-31e2-4c6d-84e1-0388a0a4cb74 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.148083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a126c89-31e2-4c6d-84e1-0388a0a4cb74 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.207791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.208388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.208673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.208947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.209574 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.211536 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.234337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1222/2481] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:15 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:15.238679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.249672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead19cc4-a26d-429e-b5eb-3c45eaf5fcfb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.255120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.255751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.256033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.256311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.256978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.258583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.270084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1260/2482] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:15.275045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.288886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c78c7-77c0-4224-930e-2521f576bf60 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.294914 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.295638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.295881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.296158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] PATH_INFO: `/v3/projects/272efe7d18184518818327ec526175c3/users/43728cef8d1e46bc8b95628406c4f86d/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.312078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272efe7d18184518818327ec526175c3, user_id=43728cef8d1e46bc8b95628406c4f86d, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.314749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.329722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1223/2483] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:15 2022] PUT /identity/v3/projects/272efe7d18184518818327ec526175c3/users/43728cef8d1e46bc8b95628406c4f86d/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:15.333855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.344149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4e9b9-4521-4d0c-9c5c-99196aadb97c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.412202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52489a2-f204-43b3-b4d5-e43891bfa480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.412830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52489a2-f204-43b3-b4d5-e43891bfa480 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.413145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52489a2-f204-43b3-b4d5-e43891bfa480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.413418 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52489a2-f204-43b3-b4d5-e43891bfa480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.414073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52489a2-f204-43b3-b4d5-e43891bfa480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.415759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52489a2-f204-43b3-b4d5-e43891bfa480 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.427301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1261/2484] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:15.431583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.443651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf17a36-e3a1-4ef3-ab13-a115f3be3d14 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.449288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.449947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.450222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.450521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] PATH_INFO: `/v3/projects/272efe7d18184518818327ec526175c3/users/43728cef8d1e46bc8b95628406c4f86d/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.452670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272efe7d18184518818327ec526175c3, user_id=43728cef8d1e46bc8b95628406c4f86d, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.455341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.466460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1224/2485] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:15 2022] PUT /identity/v3/projects/272efe7d18184518818327ec526175c3/users/43728cef8d1e46bc8b95628406c4f86d/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:15.476958 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee53b40-47f9-46ed-b7f1-42beae47a128 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.477240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee53b40-47f9-46ed-b7f1-42beae47a128 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.477623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee53b40-47f9-46ed-b7f1-42beae47a128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.513688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2ee53b40-47f9-46ed-b7f1-42beae47a128 None None] MFA Rules not processed for user `43728cef8d1e46bc8b95628406c4f86d`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:15.514936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2ee53b40-47f9-46ed-b7f1-42beae47a128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:15.543420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:15.592752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1262/2486] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:15.604320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.606255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:15.652170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:15.669562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a4df3-f580-4154-b8e7-10b2281bbc34 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.673165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.673779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.674058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.674381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.683534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.684982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.695318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1225/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:15.788268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee53b40-47f9-46ed-b7f1-42beae47a128 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:15.798331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee53b40-47f9-46ed-b7f1-42beae47a128 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:15.802119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:15.802721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:15.803001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:15.803306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:15.871838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:15.873292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:15.883073 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1263/2488] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:19.253196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.254637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.313299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e7104-8cfa-4bac-8c03-c1d8fb8aeb05 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.317146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e4232-0cf8-460b-8f06-c48cd7ad2687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.317786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4232-0cf8-460b-8f06-c48cd7ad2687 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.318072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4232-0cf8-460b-8f06-c48cd7ad2687 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.318349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4232-0cf8-460b-8f06-c48cd7ad2687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.320194 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-018e4232-0cf8-460b-8f06-c48cd7ad2687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.382946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e4232-0cf8-460b-8f06-c48cd7ad2687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.384334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e4232-0cf8-460b-8f06-c48cd7ad2687 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.395320 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1226/2489] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:19.741139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.754956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aec13-55e1-46f8-9f4b-2b25257b1f1e service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.761543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.762246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.762599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.762951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] PATH_INFO: `/v3/users/dc822e8774a14700a31f352ee7d225c8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.764201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc822e8774a14700a31f352ee7d225c8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.766303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.819461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc822e8774a14700a31f352ee7d225c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.820743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc822e8774a14700a31f352ee7d225c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.828433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc822e8774a14700a31f352ee7d225c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.829218 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc822e8774a14700a31f352ee7d225c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.839438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc822e8774a14700a31f352ee7d225c8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.858439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1264/2490] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:19 2022] DELETE /identity/v3/users/dc822e8774a14700a31f352ee7d225c8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:19.863765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.864065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.864345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.885136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:19.886311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.949782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1227/2491] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:19.959670 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.961460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.997508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:20.022420 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e4623-9b58-4f91-a018-18d5adba5932 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.026089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.026676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.026951 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.027532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.036705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.038311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.048440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1265/2492] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:20.344616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.346600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:20.367612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b580f-9e0d-4759-a629-b14c25320433 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.372550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.391343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.392064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.392557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] PATH_INFO: `/v3/projects/f0e866a421fc48c4b81edf38afcb5efc` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.402206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e866a421fc48c4b81edf38afcb5efc)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.404233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.464099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:20.484987 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e866a421fc48c4b81edf38afcb5efc', 'request_id': 'req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.496888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e866a421fc48c4b81edf38afcb5efc', 'request_id': 'req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.505004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e866a421fc48c4b81edf38afcb5efc', 'request_id': 'req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.521870 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] The token cache is being invalidate because project f0e866a421fc48c4b81edf38afcb5efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:20.522218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.523289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.524771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1228/2493] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:20 2022] DELETE /identity/v3/projects/f0e866a421fc48c4b81edf38afcb5efc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:20.528748 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.539753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dafb2-9e36-4eed-b7b4-6d2486ac9550 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.610077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.610685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.611022 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.611324 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] PATH_INFO: `/v3/users/43728cef8d1e46bc8b95628406c4f86d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.613230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] RBAC: Authorizing `identity:delete_user(user_id=43728cef8d1e46bc8b95628406c4f86d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.615005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.671170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43728cef8d1e46bc8b95628406c4f86d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.671906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43728cef8d1e46bc8b95628406c4f86d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.679348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43728cef8d1e46bc8b95628406c4f86d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.680040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43728cef8d1e46bc8b95628406c4f86d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.688861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-61149528-938c-4a7f-95fc-da890bc816df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43728cef8d1e46bc8b95628406c4f86d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:20.707019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1266/2494] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:20 2022] DELETE /identity/v3/users/43728cef8d1e46bc8b95628406c4f86d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:21.163168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.164671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:21.188346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767e511-5e85-4bbb-bb76-b3707f6fe3e3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.282054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.282738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.283106 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.283426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] PATH_INFO: `/v3/projects/272efe7d18184518818327ec526175c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.284846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=272efe7d18184518818327ec526175c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.286831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.341873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272efe7d18184518818327ec526175c3', 'request_id': 'req-629c0028-0891-4df8-9fdf-55798bb7f956'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.348164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272efe7d18184518818327ec526175c3', 'request_id': 'req-629c0028-0891-4df8-9fdf-55798bb7f956'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.354607 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272efe7d18184518818327ec526175c3', 'request_id': 'req-629c0028-0891-4df8-9fdf-55798bb7f956'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.369916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] The token cache is being invalidate because project 272efe7d18184518818327ec526175c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:21.370213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.370993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-629c0028-0891-4df8-9fdf-55798bb7f956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.372541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1229/2495] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:21 2022] DELETE /identity/v3/projects/272efe7d18184518818327ec526175c3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:21.377720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.378227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.378467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.400533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:21.401782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:21.476470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:21.494941 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1267/2496] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:21.503059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce752-9fad-47f9-9084-96daa2333574 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.503387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce752-9fad-47f9-9084-96daa2333574 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.503670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce752-9fad-47f9-9084-96daa2333574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.524486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-729ce752-9fad-47f9-9084-96daa2333574 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:21.525830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-729ce752-9fad-47f9-9084-96daa2333574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:21.535033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.535251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1230/2497] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:21.537025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:21.540180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ce752-9fad-47f9-9084-96daa2333574 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.553197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ebd71-048c-4ed6-be16-7f398805e3c7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.556077 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ce752-9fad-47f9-9084-96daa2333574 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.559236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.559898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.560544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.560822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] PATH_INFO: `/v3/users/18bc1fbb50e643918a1e5d5d1c3e8700` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.561981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18bc1fbb50e643918a1e5d5d1c3e8700)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.563293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.563970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.564090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.565003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.565379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.566347 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:21.566753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.569691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.584563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1231/2498] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:21.592656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.603943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678f5e6-d726-479d-a85d-0080e6058595 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.610235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.610848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.611138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.611451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.612412 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.614397 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.630786 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18bc1fbb50e643918a1e5d5d1c3e8700'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.631543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18bc1fbb50e643918a1e5d5d1c3e8700'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.640963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18bc1fbb50e643918a1e5d5d1c3e8700'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.641813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18bc1fbb50e643918a1e5d5d1c3e8700'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.649996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1232/2499] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:21 2022] POST /identity/v3/projects => generated 409 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:21.652437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f56ae731-1f48-48b8-aad2-d9ed58c440a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18bc1fbb50e643918a1e5d5d1c3e8700'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:21.652961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.654773 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:21.661332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:21.675994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1268/2500] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:21 2022] DELETE /identity/v3/users/18bc1fbb50e643918a1e5d5d1c3e8700 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:21.680204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.680993 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.681353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.703315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:21.704581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:21.734285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db8c7e-9019-482c-b1d1-39edc9a57972 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.776657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1269/2501] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:21.787892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.789527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:21.815925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.816945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.817422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.818006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.818732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.821109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.852134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1233/2502] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:21 2022] POST /identity/v3/users => generated 305 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:21.856629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.866343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c7bca-3b2d-4e2e-bbdf-c707e5c10bc3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.868168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c130c9-d1a0-44cf-8484-7612515d55dd admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.872103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.873198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.873519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.873823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.874252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.874498 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.874881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.875302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.876325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.878467 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.893277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.895618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.896998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1234/2503] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:21.898063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.908772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49211fd6-4164-4dee-a4cb-5c3cb1e34535 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.914247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a645ab-f0c9-43b7-8c7d-df6b26cc3976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.914999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a645ab-f0c9-43b7-8c7d-df6b26cc3976 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.915423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a645ab-f0c9-43b7-8c7d-df6b26cc3976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.915801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a645ab-f0c9-43b7-8c7d-df6b26cc3976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.916686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a645ab-f0c9-43b7-8c7d-df6b26cc3976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.916967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1270/2504] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:21.918538 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a645ab-f0c9-43b7-8c7d-df6b26cc3976 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.930749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1235/2505] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:21.935875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.949473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc90a18-ea78-40cf-bea3-573a54c3cce4 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.962829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.963599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.963956 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.964854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] PATH_INFO: `/v3/projects/1a59b0a87eee46c1b1c132160d910fa4/users/45d2fed09ea74fb39139d11f88cf257a/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.968118 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:21.987134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a59b0a87eee46c1b1c132160d910fa4, user_id=45d2fed09ea74fb39139d11f88cf257a, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.991625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.008090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1271/2506] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:21 2022] PUT /identity/v3/projects/1a59b0a87eee46c1b1c132160d910fa4/users/45d2fed09ea74fb39139d11f88cf257a/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:22.020095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be97fc0-9c53-45ce-82b2-114b3494c1d6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.020384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be97fc0-9c53-45ce-82b2-114b3494c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.020663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be97fc0-9c53-45ce-82b2-114b3494c1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.050497 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4be97fc0-9c53-45ce-82b2-114b3494c1d6 None None] MFA Rules not processed for user `45d2fed09ea74fb39139d11f88cf257a`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:22.051911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4be97fc0-9c53-45ce-82b2-114b3494c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:22.139251 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1236/2507] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:22.144929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.214821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53588b94-f3c0-45e2-8427-d00530f05086 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.219081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.219733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.220016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.220287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.221196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.223227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.252287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1272/2508] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:22.257140 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be97fc0-9c53-45ce-82b2-114b3494c1d6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.274660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be97fc0-9c53-45ce-82b2-114b3494c1d6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.286725 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:22.299628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.302028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:22.374823 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.375658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.376120 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.376488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.377352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.380767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.385089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303ba26-65ea-4ee3-a2ae-1750ed99bf93 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.389971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.390736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.391112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.391614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] PATH_INFO: `/v3/projects/cba7269cffa2467b9a1a13180d943207` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.393631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba7269cffa2467b9a1a13180d943207)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.396689 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.410556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1237/2509] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/users => generated 310 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:22.415378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.430980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca83e-f670-41e9-8dce-9228ffa504b5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.438713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.439532 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.440214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.440406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.441151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.442868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.454901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1238/2510] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:22.462336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.476674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689aea1-9a5f-4556-9ec5-c1a127ba1572 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.483914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba7269cffa2467b9a1a13180d943207', 'request_id': 'req-df3668d4-6a14-4083-9d83-3a98dea73633'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:22.493973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba7269cffa2467b9a1a13180d943207', 'request_id': 'req-df3668d4-6a14-4083-9d83-3a98dea73633'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:22.502065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba7269cffa2467b9a1a13180d943207', 'request_id': 'req-df3668d4-6a14-4083-9d83-3a98dea73633'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:22.519570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:22.530222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] The token cache is being invalidate because project cba7269cffa2467b9a1a13180d943207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:22.530960 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:22.533040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-df3668d4-6a14-4083-9d83-3a98dea73633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:22.535679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1273/2511] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:22 2022] DELETE /identity/v3/projects/cba7269cffa2467b9a1a13180d943207 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:22.542613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25338109-db3a-4e60-bae0-1bf098b53297 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.543912 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25338109-db3a-4e60-bae0-1bf098b53297 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.544347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25338109-db3a-4e60-bae0-1bf098b53297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.574340 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.575246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-25338109-db3a-4e60-bae0-1bf098b53297 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:22.576500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-25338109-db3a-4e60-bae0-1bf098b53297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:22.585502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.585772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.586109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] PATH_INFO: `/v3/projects/658028eff0464b4880b4d4bf6fafc240/users/714f5b670d7c45b08fad5abaa70b25a6/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.588266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1274/2512] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:22.596145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04318d21-3b67-4f54-9bae-851a2c925625 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.596744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04318d21-3b67-4f54-9bae-851a2c925625 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.597205 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04318d21-3b67-4f54-9bae-851a2c925625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.608543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] RBAC: Authorizing `identity:create_grant(project_id=658028eff0464b4880b4d4bf6fafc240, user_id=714f5b670d7c45b08fad5abaa70b25a6, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.612139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.631037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-04318d21-3b67-4f54-9bae-851a2c925625 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:22.632882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-04318d21-3b67-4f54-9bae-851a2c925625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:22.636037 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1239/2513] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:22 2022] PUT /identity/v3/projects/658028eff0464b4880b4d4bf6fafc240/users/714f5b670d7c45b08fad5abaa70b25a6/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:22.639712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.644913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:22.744410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1275/2514] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:22.751996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04318d21-3b67-4f54-9bae-851a2c925625 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.764224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04318d21-3b67-4f54-9bae-851a2c925625 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.767403 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305d43d-f259-4e47-9e63-f97a0e9b964d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.769677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.770675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.771018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.771385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.772277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:22.772646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.774266 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.774516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.775029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.775371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.775652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.776315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.778898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.788902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1276/2515] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:22.794387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.798642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1240/2516] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:22.807076 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.813979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e660ade-f255-43c3-a5d3-1a423a119e73 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.819511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.820431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.820916 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.821416 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.822394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.823927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca61c4-f18c-49f7-863b-4c742194d9cb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:22.824862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.835450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020c754-b5b8-43db-90c9-a9c0f4bbf1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:22.836535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020c754-b5b8-43db-90c9-a9c0f4bbf1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.836900 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020c754-b5b8-43db-90c9-a9c0f4bbf1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.837328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020c754-b5b8-43db-90c9-a9c0f4bbf1c8 admin admin] PATH_INFO: `/v3/projects/658028eff0464b4880b4d4bf6fafc240/users/714f5b670d7c45b08fad5abaa70b25a6/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.839868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020c754-b5b8-43db-90c9-a9c0f4bbf1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658028eff0464b4880b4d4bf6fafc240, user_id=714f5b670d7c45b08fad5abaa70b25a6, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:22.847645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020c754-b5b8-43db-90c9-a9c0f4bbf1c8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:22.851156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:22.873605 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1241/2517] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:22 2022] PUT /identity/v3/projects/658028eff0464b4880b4d4bf6fafc240/users/714f5b670d7c45b08fad5abaa70b25a6/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:22.885062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:22.885347 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:22.885624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:22.890657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1277/2518] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:22.892255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:22.921012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] MFA Rules not processed for user `714f5b670d7c45b08fad5abaa70b25a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:22.922285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:22.997406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dd33c-03af-45b3-b0ea-1004b40db62b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.003072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.003868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.004750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.005139 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.006090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.008248 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.022372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1242/2519] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:23.034146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.035787 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.038625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1278/2520] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:22 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:23.043649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.056171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85865385-b71f-4b23-9846-b58fd8d4d54c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.061810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.062474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.062761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.063032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.063724 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.065774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.079458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1279/2521] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:23.080590 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.091643 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ea29b-bb54-461b-990b-9470f20e5b7e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.098460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.099247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.099542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.099821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.100527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.102146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.117274 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d488c-e9e1-498a-9a91-9eaeb826bbab None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.118316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1280/2522] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:23.123402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.129401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.130126 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.130667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.131048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.137124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6521a5c-1490-4dab-a69f-b29bcffdcba2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.142705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d1411-5a62-463a-a951-dfc728ef479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.143778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1411-5a62-463a-a951-dfc728ef479b admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.144238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.144367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1411-5a62-463a-a951-dfc728ef479b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.144652 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d1411-5a62-463a-a951-dfc728ef479b admin admin] PATH_INFO: `/v3/projects/13dae75bec47489cac0248f2bb2c197d/users/0b23553f7543446cb5eca18f26e25816/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.145715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.155378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1243/2523] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:23 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:23.156264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.157620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.162225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d1411-5a62-463a-a951-dfc728ef479b admin admin] RBAC: Authorizing `identity:create_grant(project_id=13dae75bec47489cac0248f2bb2c197d, user_id=0b23553f7543446cb5eca18f26e25816, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.164816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d1411-5a62-463a-a951-dfc728ef479b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.172836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e04dc-8dc2-488a-ab94-737507cfcc45 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.177675 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.178347 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.178633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.178902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] PATH_INFO: `/v3/users/f264e354671d4456bb33668fb11d9636` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.180268 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f264e354671d4456bb33668fb11d9636)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.181511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1281/2524] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:23 2022] PUT /identity/v3/projects/13dae75bec47489cac0248f2bb2c197d/users/0b23553f7543446cb5eca18f26e25816/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:23.182369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.187061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.187363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.187626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.210446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:23.234708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:23.236036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.243318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f264e354671d4456bb33668fb11d9636'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.250724 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f264e354671d4456bb33668fb11d9636'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.261422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f264e354671d4456bb33668fb11d9636'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.262350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f264e354671d4456bb33668fb11d9636'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.263217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f264e354671d4456bb33668fb11d9636'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.277656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:23.287441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1244/2525] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:23 2022] DELETE /identity/v3/users/f264e354671d4456bb33668fb11d9636 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:23.316342 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1282/2526] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:23.327530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.329206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.351490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.353049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.400449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da898-82f0-47b2-bccf-38297fae17fd admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.405622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.406488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.406779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.407056 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.419821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed165c-f45c-4c95-b7ec-42c6f686e84a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.495886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bb277-ba91-4bf7-b18c-7e330bc344c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.496525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb277-ba91-4bf7-b18c-7e330bc344c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.496877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb277-ba91-4bf7-b18c-7e330bc344c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.497132 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb277-ba91-4bf7-b18c-7e330bc344c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.498991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-389bb277-ba91-4bf7-b18c-7e330bc344c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.508681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.510363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.521170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1245/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:23.522951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.525275 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.550074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba36b-ecb6-444a-a708-3bd6e59b58be service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.557523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.558663 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.559254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.559374 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] PATH_INFO: `/v3/projects/a41fb206b09941ac80996a3c9cbaae58` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.562145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41fb206b09941ac80996a3c9cbaae58)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.565164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.581062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bb277-ba91-4bf7-b18c-7e330bc344c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.583108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bb277-ba91-4bf7-b18c-7e330bc344c3 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.595119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1283/2528] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:23 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:23.634780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41fb206b09941ac80996a3c9cbaae58', 'request_id': 'req-38643866-2182-4a3e-9ec4-9b17ae3850a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.642294 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41fb206b09941ac80996a3c9cbaae58', 'request_id': 'req-38643866-2182-4a3e-9ec4-9b17ae3850a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.650173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41fb206b09941ac80996a3c9cbaae58', 'request_id': 'req-38643866-2182-4a3e-9ec4-9b17ae3850a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.665596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] The token cache is being invalidate because project a41fb206b09941ac80996a3c9cbaae58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:23.665892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.666653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-38643866-2182-4a3e-9ec4-9b17ae3850a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:23.668772 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1246/2529] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:23 2022] DELETE /identity/v3/projects/a41fb206b09941ac80996a3c9cbaae58 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:23.673572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e011c-a2f8-4cbc-b408-32030051a8af None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.673901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e011c-a2f8-4cbc-b408-32030051a8af None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.675503 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e011c-a2f8-4cbc-b408-32030051a8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.698735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-216e011c-a2f8-4cbc-b408-32030051a8af None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:23.700009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-216e011c-a2f8-4cbc-b408-32030051a8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.712011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:23.783048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1284/2530] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:23.785047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4557-ea24-4add-8ea2-1525b7b607e3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.785333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4557-ea24-4add-8ea2-1525b7b607e3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.785613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f4557-ea24-4add-8ea2-1525b7b607e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.808224 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c59f4557-ea24-4add-8ea2-1525b7b607e3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:23.809591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c59f4557-ea24-4add-8ea2-1525b7b607e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:23.820483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1247/2531] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:23.824763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e011c-a2f8-4cbc-b408-32030051a8af None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.839322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e011c-a2f8-4cbc-b408-32030051a8af None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.849900 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.850925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.851429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.851892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.853002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:23.853519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.856400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.875170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1285/2532] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:23.879758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4557-ea24-4add-8ea2-1525b7b607e3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.893493 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f4557-ea24-4add-8ea2-1525b7b607e3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:23.903910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:23.904634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:23.904932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:23.905350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:23.906101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:23.909067 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:23.936817 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:23.948494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1248/2533] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:23 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:23.953597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:23.969461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe541a-a74a-4675-ba0b-6e8763f554b9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.057204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.057852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.058176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.058439 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.059033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.061426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.090366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1286/2534] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:23 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:24.094801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.106465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad4032-0be3-4775-a619-2f4a90620dd1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.111244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.111930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.112424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.112824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.113475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.115089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.125316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1249/2535] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:24.129885 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.140961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f8941-5542-4977-acde-eda8b4040277 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.147667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b3db0-519f-46eb-9e02-cc3e0f483d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.148293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b3db0-519f-46eb-9e02-cc3e0f483d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.148589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b3db0-519f-46eb-9e02-cc3e0f483d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.148856 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b3db0-519f-46eb-9e02-cc3e0f483d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.149540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b3db0-519f-46eb-9e02-cc3e0f483d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.151214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b3db0-519f-46eb-9e02-cc3e0f483d8e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.162465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1287/2536] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:24.166801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.177635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1107a2-bfd7-4588-bbb7-ba4fffa24e57 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.182691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.183298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.183584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.184008 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] PATH_INFO: `/v3/projects/7cabda88d2e041879a46ca4490e375fe/users/0c92f8a34ba4464582f34a01bf9595cf/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.200530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cabda88d2e041879a46ca4490e375fe, user_id=0c92f8a34ba4464582f34a01bf9595cf, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.203433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.218739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1250/2537] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:24 2022] PUT /identity/v3/projects/7cabda88d2e041879a46ca4490e375fe/users/0c92f8a34ba4464582f34a01bf9595cf/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:24.229876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bd9f8-3096-4f4f-b27c-b4c87359a7b4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.230495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bd9f8-3096-4f4f-b27c-b4c87359a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.230495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bd9f8-3096-4f4f-b27c-b4c87359a7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.262306 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3b1bd9f8-3096-4f4f-b27c-b4c87359a7b4 None None] MFA Rules not processed for user `0c92f8a34ba4464582f34a01bf9595cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:24.263560 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3b1bd9f8-3096-4f4f-b27c-b4c87359a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:24.340466 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1288/2538] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:24.345164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.414659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c7e3-a878-49cc-b00e-3e19461ed883 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.419629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.420416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.420767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.421109 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.421843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.423904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.457448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1251/2539] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:24 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:24.461811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bd9f8-3096-4f4f-b27c-b4c87359a7b4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.475669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bd9f8-3096-4f4f-b27c-b4c87359a7b4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.482045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:24.546294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.547094 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.547923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.547923 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.548692 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.551049 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.576077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1289/2540] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:24 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:24.580550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.591927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbebc0-f9c1-4e1f-a938-1fa52cca77bf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.597305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.598932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.598932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.598932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.599766 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.601523 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.617648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1252/2541] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:24.621732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.633468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d35108-19e0-471d-be22-2bcf0708fe7f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.638735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.639602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.639985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.640362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] PATH_INFO: `/v3/projects/02ed1aaab90146d3a57d0eb4f5aa3fb9/users/80c10a08ffdd477d89ec5ddc08df11bc/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.660696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ed1aaab90146d3a57d0eb4f5aa3fb9, user_id=80c10a08ffdd477d89ec5ddc08df11bc, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.663917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.680893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1290/2542] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:24 2022] PUT /identity/v3/projects/02ed1aaab90146d3a57d0eb4f5aa3fb9/users/80c10a08ffdd477d89ec5ddc08df11bc/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:24.685800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.698390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed41dd-5dd9-4edc-ac2e-ac591d04c11a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.767062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f656b-852a-4b89-9976-34135b64a4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.768303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f656b-852a-4b89-9976-34135b64a4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.768566 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f656b-852a-4b89-9976-34135b64a4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.768943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f656b-852a-4b89-9976-34135b64a4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.769911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f656b-852a-4b89-9976-34135b64a4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.771668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f656b-852a-4b89-9976-34135b64a4f6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.786563 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1253/2543] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:24.791108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:24.802037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffb609-6757-42e6-a5ee-c149f78290cd admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:24.806651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:24.807362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.807646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.807922 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] PATH_INFO: `/v3/projects/02ed1aaab90146d3a57d0eb4f5aa3fb9/users/80c10a08ffdd477d89ec5ddc08df11bc/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.809842 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ed1aaab90146d3a57d0eb4f5aa3fb9, user_id=80c10a08ffdd477d89ec5ddc08df11bc, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:24.813005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:24.824413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1291/2544] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:24 2022] PUT /identity/v3/projects/02ed1aaab90146d3a57d0eb4f5aa3fb9/users/80c10a08ffdd477d89ec5ddc08df11bc/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:24.836826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dff83-7078-4012-8320-cf6eded6ca4c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:24.837135 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dff83-7078-4012-8320-cf6eded6ca4c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:24.837424 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dff83-7078-4012-8320-cf6eded6ca4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:24.873223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-583dff83-7078-4012-8320-cf6eded6ca4c None None] MFA Rules not processed for user `80c10a08ffdd477d89ec5ddc08df11bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:24.875346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-583dff83-7078-4012-8320-cf6eded6ca4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:24.916762 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:24.987856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1254/2545] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:24.998625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:25.000532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:25.132488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5e3ab-5f90-4b2c-8911-ec413540ec4b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:25.143012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2bf68-fdd3-4a4a-b662-af970b4bbea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:25.145488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2bf68-fdd3-4a4a-b662-af970b4bbea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:25.146552 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2bf68-fdd3-4a4a-b662-af970b4bbea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:25.147013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2bf68-fdd3-4a4a-b662-af970b4bbea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:25.164564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2bf68-fdd3-4a4a-b662-af970b4bbea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:25.168470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2bf68-fdd3-4a4a-b662-af970b4bbea4 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:25.188579 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1292/2546] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:25.275520 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dff83-7078-4012-8320-cf6eded6ca4c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:25.293867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dff83-7078-4012-8320-cf6eded6ca4c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:25.298026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:25.298910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:25.299825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:25.300421 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:25.451020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:25.452706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:25.491331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1255/2547] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:27.957409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc0afd-ac68-4b57-9cf5-a0f4625a30a4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:27.957934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc0afd-ac68-4b57-9cf5-a0f4625a30a4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:27.957934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc0afd-ac68-4b57-9cf5-a0f4625a30a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:27.986942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7bfc0afd-ac68-4b57-9cf5-a0f4625a30a4 None None] MFA Rules not processed for user `0b23553f7543446cb5eca18f26e25816`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:27.988223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7bfc0afd-ac68-4b57-9cf5-a0f4625a30a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:28.044894 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:28.059330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1293/2548] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:28.063535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:28.065291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:28.149804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6286a3d-2453-465b-ac6d-032c29cdd967 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:28.154620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:28.155352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:28.155699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:28.156044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:28.156789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:28.158884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:28.193449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1256/2549] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:28 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:28.198001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc0afd-ac68-4b57-9cf5-a0f4625a30a4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:28.215824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc0afd-ac68-4b57-9cf5-a0f4625a30a4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:28.294616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:28.295299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:28.295585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:28.295855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:28.296519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:28.298515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:28.328662 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1294/2550] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:28 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:28.332721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:28.345636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298615a-465c-49a5-9291-88368da2007f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:28.355108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:28.355773 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:28.356058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:28.356334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:28.357055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:28.358669 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:28.401443 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1257/2551] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:28.402642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:28.416683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c510c-3401-4218-942b-992338595c6a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:28.421822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:28.422498 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:28.422787 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:28.423066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] PATH_INFO: `/v3/projects/9bc1ce04283647c5a0693deee538d90d/users/6d2c1bdf17fd485fb59273543850f2eb/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:28.456298 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc1ce04283647c5a0693deee538d90d, user_id=6d2c1bdf17fd485fb59273543850f2eb, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:28.462034 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:28.493248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:28.493771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1295/2552] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:28 2022] PUT /identity/v3/projects/9bc1ce04283647c5a0693deee538d90d/users/6d2c1bdf17fd485fb59273543850f2eb/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:28.507042 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32638d-b02e-4efc-96a5-aeb809fd88c2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:28.588864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:28.645406 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:28.646170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:28.646504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:28.646754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:28.647508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:28.649151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:28.667290 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1258/2553] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:28.670250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:28.689573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191d910-b1c2-4ac2-baaa-c78e16f2f615 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:28.704040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc60e5-e0f0-44a4-88f7-08e28f07d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:28.704735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc60e5-e0f0-44a4-88f7-08e28f07d205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:28.705014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc60e5-e0f0-44a4-88f7-08e28f07d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:28.705354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc60e5-e0f0-44a4-88f7-08e28f07d205 admin admin] PATH_INFO: `/v3/projects/9bc1ce04283647c5a0693deee538d90d/users/6d2c1bdf17fd485fb59273543850f2eb/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:28.708187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc60e5-e0f0-44a4-88f7-08e28f07d205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc1ce04283647c5a0693deee538d90d, user_id=6d2c1bdf17fd485fb59273543850f2eb, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:28.711097 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc60e5-e0f0-44a4-88f7-08e28f07d205 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:28.738108 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1296/2554] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:28 2022] PUT /identity/v3/projects/9bc1ce04283647c5a0693deee538d90d/users/6d2c1bdf17fd485fb59273543850f2eb/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:29.457093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:29.474334 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7804fd-3f0c-412a-9c35-cf36d81574e9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:29.549481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:29.550143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:29.550432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:29.550713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] PATH_INFO: `/v3/users/0c92f8a34ba4464582f34a01bf9595cf` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:29.552483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c92f8a34ba4464582f34a01bf9595cf)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:29.554634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:29.630901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c92f8a34ba4464582f34a01bf9595cf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:29.639641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c92f8a34ba4464582f34a01bf9595cf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:29.652819 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c92f8a34ba4464582f34a01bf9595cf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:29.653541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c92f8a34ba4464582f34a01bf9595cf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:29.654394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c92f8a34ba4464582f34a01bf9595cf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:29.683996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1259/2555] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:29 2022] DELETE /identity/v3/users/0c92f8a34ba4464582f34a01bf9595cf => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:29.686604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:29.687128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:29.687364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:29.713451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:29.714782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:29.814561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1297/2556] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:29.823015 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:29.826449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:29.936945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:29.944066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04d327-6b96-4326-900c-78ce55083306 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:29.949866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:29.950842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:29.951379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:29.951932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:29.966814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:29.969279 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:29.982833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1260/2557] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:31.087141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:31.089427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:31.102625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dad106-a42d-4fa5-9732-92e5053b33ca None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:31.114355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:31.115101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:31.116225 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:31.116567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] PATH_INFO: `/v3/projects/7cabda88d2e041879a46ca4490e375fe` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:31.117964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cabda88d2e041879a46ca4490e375fe)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:31.120697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:31.202665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:31.211612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cabda88d2e041879a46ca4490e375fe', 'request_id': 'req-07067cad-2c48-4f28-b639-89fd371b02e1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.221534 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cabda88d2e041879a46ca4490e375fe', 'request_id': 'req-07067cad-2c48-4f28-b639-89fd371b02e1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.229917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cabda88d2e041879a46ca4490e375fe', 'request_id': 'req-07067cad-2c48-4f28-b639-89fd371b02e1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.257013 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] The token cache is being invalidate because project 7cabda88d2e041879a46ca4490e375fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:31.257320 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.258027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.263138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1298/2558] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:31 2022] DELETE /identity/v3/projects/7cabda88d2e041879a46ca4490e375fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:31.265801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:31.277590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0b0ca-95d4-4296-84d1-ec0083461445 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:31.351006 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:31.353312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:31.353598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:31.354213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] PATH_INFO: `/v3/users/80c10a08ffdd477d89ec5ddc08df11bc` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:31.355284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c10a08ffdd477d89ec5ddc08df11bc)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:31.357575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:31.475238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c10a08ffdd477d89ec5ddc08df11bc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.496227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c10a08ffdd477d89ec5ddc08df11bc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.515293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c10a08ffdd477d89ec5ddc08df11bc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.515293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c10a08ffdd477d89ec5ddc08df11bc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.518417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c10a08ffdd477d89ec5ddc08df11bc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:31.565402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1261/2559] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:31 2022] DELETE /identity/v3/users/80c10a08ffdd477d89ec5ddc08df11bc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:32.006191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:32.012175 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:32.058830 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07067cad-2c48-4f28-b639-89fd371b02e1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:32.251516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:32.252659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.253475 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.253836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] PATH_INFO: `/v3/users/45d2fed09ea74fb39139d11f88cf257a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.255924 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=45d2fed09ea74fb39139d11f88cf257a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:32.258060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:32.327028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:32.328586 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:32.342440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45d2fed09ea74fb39139d11f88cf257a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.343305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45d2fed09ea74fb39139d11f88cf257a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.355742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3639bc-6a08-4eab-89f2-c7e3bdabcacf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:32.356371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45d2fed09ea74fb39139d11f88cf257a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.357187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45d2fed09ea74fb39139d11f88cf257a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.367886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:32.368252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-abaeb1b3-7a0d-4a09-bc39-f1546ef75dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45d2fed09ea74fb39139d11f88cf257a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.368558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.368842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.369163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] PATH_INFO: `/v3/projects/02ed1aaab90146d3a57d0eb4f5aa3fb9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.371719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ed1aaab90146d3a57d0eb4f5aa3fb9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:32.373640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:32.390311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1299/2560] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:32 2022] DELETE /identity/v3/users/45d2fed09ea74fb39139d11f88cf257a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:32.395244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.397566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.397902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.460463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ed1aaab90146d3a57d0eb4f5aa3fb9', 'request_id': 'req-946e30e9-305e-4a03-ba67-c0fdab3190ee'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.461394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:32.462853 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:32.473653 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ed1aaab90146d3a57d0eb4f5aa3fb9', 'request_id': 'req-946e30e9-305e-4a03-ba67-c0fdab3190ee'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.498961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ed1aaab90146d3a57d0eb4f5aa3fb9', 'request_id': 'req-946e30e9-305e-4a03-ba67-c0fdab3190ee'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.503386 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:32.530629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] The token cache is being invalidate because project 02ed1aaab90146d3a57d0eb4f5aa3fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:32.530972 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.531960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-946e30e9-305e-4a03-ba67-c0fdab3190ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:32.535463 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1262/2561] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:32 2022] DELETE /identity/v3/projects/02ed1aaab90146d3a57d0eb4f5aa3fb9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:32.540938 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddcf5a-cda1-4f83-8199-1776d1042c18 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.541903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddcf5a-cda1-4f83-8199-1776d1042c18 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.543054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddcf5a-cda1-4f83-8199-1776d1042c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.577455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:32.598501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4eddcf5a-cda1-4f83-8199-1776d1042c18 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:32.599790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4eddcf5a-cda1-4f83-8199-1776d1042c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:32.618284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1300/2562] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:32.632178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:32.633609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1263/2563] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:32.633925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:32.643022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7a1fe-1ade-46f6-9971-d41e1b18c71b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.643539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7a1fe-1ade-46f6-9971-d41e1b18c71b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.643930 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7a1fe-1ade-46f6-9971-d41e1b18c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.688358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dad7a1fe-1ade-46f6-9971-d41e1b18c71b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:32.688358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dad7a1fe-1ade-46f6-9971-d41e1b18c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:32.714854 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1264/2564] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:32.719420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7a1fe-1ade-46f6-9971-d41e1b18c71b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:32.739366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7a1fe-1ade-46f6-9971-d41e1b18c71b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:32.752595 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:32.753440 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.753811 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.754164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.755351 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1ba45-5b45-470e-991c-ddfe7ca03a63 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:32.755489 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:32.757813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:32.759734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:32.762084 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:32.763396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.764037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.764742 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.780165 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:32.782107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:32.795541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1265/2565] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:32.797368 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:32.811355 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1301/2566] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:32.822096 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12a2a2-807f-4da5-9568-160a49680e31 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:32.838591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:32.839401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.839758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.840273 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.840986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:32.843030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:32.893462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1266/2567] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:32 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:32.897705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:32.912207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b7de8-a15e-4692-bbff-256516b29f42 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:33.047290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:33.047934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.048214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.048487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.049925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:33.051977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:33.109060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1302/2568] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:32 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:33.114746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:33.129590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5b9d8-56a0-4177-b82c-30fac672be35 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:33.140820 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:33.143671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.144172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.144601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.146204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:33.150508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:33.187901 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1267/2569] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:33.190119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:33.208199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da465e68-aadc-4c82-b424-0a874ac58a45 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:33.225236 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:33.226150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.226648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.227077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.227938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:33.229721 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:33.243259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1303/2570] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:33.247965 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:33.262234 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f3b2f-0b50-4b26-bcb7-c62571a29be0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:33.268696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1f8d9-bd3b-4ed1-aece-642aa0003a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:33.270395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1f8d9-bd3b-4ed1-aece-642aa0003a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.271038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1f8d9-bd3b-4ed1-aece-642aa0003a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.271038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1f8d9-bd3b-4ed1-aece-642aa0003a9f admin admin] PATH_INFO: `/v3/projects/32e331d852ea4fe397957b8e5a14fb06/users/f38eb17e93c14f4da999c1df2440d41c/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.294673 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1f8d9-bd3b-4ed1-aece-642aa0003a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e331d852ea4fe397957b8e5a14fb06, user_id=f38eb17e93c14f4da999c1df2440d41c, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:33.298401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1f8d9-bd3b-4ed1-aece-642aa0003a9f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:33.311086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:33.314456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:33.350265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1268/2571] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:33 2022] PUT /identity/v3/projects/32e331d852ea4fe397957b8e5a14fb06/users/f38eb17e93c14f4da999c1df2440d41c/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:33.350861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b979ac-ecc9-40e4-bbb8-6be7aba76f99 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:33.361124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64941477-9bd5-49ec-a72a-d1732dfab97a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.361813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64941477-9bd5-49ec-a72a-d1732dfab97a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.361813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64941477-9bd5-49ec-a72a-d1732dfab97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.409175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-64941477-9bd5-49ec-a72a-d1732dfab97a None None] MFA Rules not processed for user `f38eb17e93c14f4da999c1df2440d41c`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:33.413780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-64941477-9bd5-49ec-a72a-d1732dfab97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:33.458703 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:33.459985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.460564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.461042 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] PATH_INFO: `/v3/projects/1a59b0a87eee46c1b1c132160d910fa4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.462945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a59b0a87eee46c1b1c132160d910fa4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:33.479616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:33.516855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:33.617268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a59b0a87eee46c1b1c132160d910fa4', 'request_id': 'req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:33.621416 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1269/2572] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:33.637123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a59b0a87eee46c1b1c132160d910fa4', 'request_id': 'req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:33.637835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64941477-9bd5-49ec-a72a-d1732dfab97a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:33.651644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a59b0a87eee46c1b1c132160d910fa4', 'request_id': 'req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:33.679657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] The token cache is being invalidate because project 1a59b0a87eee46c1b1c132160d910fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:33.680032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:33.680764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:33.687945 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1304/2573] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:33 2022] DELETE /identity/v3/projects/1a59b0a87eee46c1b1c132160d910fa4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:33.692516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:33.696533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64941477-9bd5-49ec-a72a-d1732dfab97a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:33.697277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:33.826016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:33.844691 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:33.845423 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.846040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.847578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.849516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:33.852682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:33.857981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2df9ca-d77b-4e70-92f5-77dc3bec5863 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:33.875437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:33.876212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:33.877177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:33.877566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] PATH_INFO: `/v3/users/714f5b670d7c45b08fad5abaa70b25a6` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:33.880192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714f5b670d7c45b08fad5abaa70b25a6)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:33.883287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:33.915028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1270/2574] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:33 2022] POST /identity/v3/projects => generated 379 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:33.916150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:33.919447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:33.988428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714f5b670d7c45b08fad5abaa70b25a6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:33.989174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714f5b670d7c45b08fad5abaa70b25a6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:34.012758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515099d-e113-4696-b644-cb42b3bb18cc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:34.019927 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714f5b670d7c45b08fad5abaa70b25a6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:34.022231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714f5b670d7c45b08fad5abaa70b25a6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:34.034692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:34.035585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:34.035908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:34.036139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:34.036796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:34.038836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:34.069947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714f5b670d7c45b08fad5abaa70b25a6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:34.078323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1271/2575] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:33 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:34.085516 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:34.091078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:34.121130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2e6db-d5d1-49f5-816d-8e24dba3c9ee admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:34.122041 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1305/2576] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:33 2022] DELETE /identity/v3/users/714f5b670d7c45b08fad5abaa70b25a6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:34.262116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:34.263168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:34.263954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:34.264991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:34.265898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:34.270431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:34.292173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1272/2577] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:34.296349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:34.319591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fe3dd-4342-4abd-90f3-c765737f7e55 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:34.333571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e732f1-4bf5-4b5f-8d6e-2e505d9ba355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:34.334212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e732f1-4bf5-4b5f-8d6e-2e505d9ba355 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:34.334492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e732f1-4bf5-4b5f-8d6e-2e505d9ba355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:34.335232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e732f1-4bf5-4b5f-8d6e-2e505d9ba355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:34.335459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e732f1-4bf5-4b5f-8d6e-2e505d9ba355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:34.337080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e732f1-4bf5-4b5f-8d6e-2e505d9ba355 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:34.355289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1306/2578] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:34.359741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:34.375253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a3603-361b-4c17-9a10-835ff8e00600 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:34.390617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:34.391494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:34.391783 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:34.392086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] PATH_INFO: `/v3/projects/5a4704c500834cf095e982d7cc20cfd8/users/1daa433397f34fba96cbd988f54a57a8/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:34.442197 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4704c500834cf095e982d7cc20cfd8, user_id=1daa433397f34fba96cbd988f54a57a8, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:34.445719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:34.483623 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1273/2579] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:34 2022] PUT /identity/v3/projects/5a4704c500834cf095e982d7cc20cfd8/users/1daa433397f34fba96cbd988f54a57a8/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:34.497485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543858fc-5a3e-4cda-9224-91e5d74aad13 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:34.497812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543858fc-5a3e-4cda-9224-91e5d74aad13 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:34.498239 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543858fc-5a3e-4cda-9224-91e5d74aad13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:34.536526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:34.562348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-543858fc-5a3e-4cda-9224-91e5d74aad13 None None] MFA Rules not processed for user `1daa433397f34fba96cbd988f54a57a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:34.570228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-543858fc-5a3e-4cda-9224-91e5d74aad13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:34.762193 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1307/2580] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:34.763936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:34.888370 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:34.935503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51efd64-4b77-4014-b7a0-af01ef3325bc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:34.950998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:34.952326 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:34.954122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:34.955946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:34.957317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:34.962417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.041753 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1274/2581] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:34 2022] POST /identity/v3/projects => generated 377 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:35.043685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543858fc-5a3e-4cda-9224-91e5d74aad13 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.067313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543858fc-5a3e-4cda-9224-91e5d74aad13 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.183463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.184531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.185559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.185938 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.186691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.206811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.258353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1308/2582] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:35 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:35.262873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.293737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6b249-61e3-4eb8-b068-bacc223ceca3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.306711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.308915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:35.309639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.310638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.311889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.312337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.312945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.314693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.333079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388efd7-cd6a-4148-9576-e00a87c5d44e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.354076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.354521 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.359040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1275/2583] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:35.359251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.359448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] PATH_INFO: `/v3/projects/658028eff0464b4880b4d4bf6fafc240` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.360608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.362726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658028eff0464b4880b4d4bf6fafc240)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.367677 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.382134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c81b6-08f6-4a13-88a5-78137b936464 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.397211 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.397897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.398177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.398456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] PATH_INFO: `/v3/projects/fd429ab2fd1641c2b7260c59d1b3a9b1/users/a213e36d23f24c8dbc30cf2ceedc0c36/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.424826 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd429ab2fd1641c2b7260c59d1b3a9b1, user_id=a213e36d23f24c8dbc30cf2ceedc0c36, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.430845 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.457216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1276/2584] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:35 2022] PUT /identity/v3/projects/fd429ab2fd1641c2b7260c59d1b3a9b1/users/a213e36d23f24c8dbc30cf2ceedc0c36/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:35.461350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.488566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658028eff0464b4880b4d4bf6fafc240', 'request_id': 'req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:35.498460 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658028eff0464b4880b4d4bf6fafc240', 'request_id': 'req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:35.512076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658028eff0464b4880b4d4bf6fafc240', 'request_id': 'req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:35.547417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] The token cache is being invalidate because project 658028eff0464b4880b4d4bf6fafc240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:35.548085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:35.549587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a0cc53cd-1032-4e58-abdd-7ce5ccd612a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:35.552929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1309/2585] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:35 2022] DELETE /identity/v3/projects/658028eff0464b4880b4d4bf6fafc240 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:35.558789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9af20-17f2-4349-a385-ffec559a1c98 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.559428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9af20-17f2-4349-a385-ffec559a1c98 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.559948 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9af20-17f2-4349-a385-ffec559a1c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.573539 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be375c25-6088-484a-bda1-e8e92b38244d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.607508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b0e9af20-17f2-4349-a385-ffec559a1c98 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:35.609999 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b0e9af20-17f2-4349-a385-ffec559a1c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:35.616106 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:35.672129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.672770 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.673221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.673733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.674460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.676864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.692915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1277/2586] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:35.699633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.701557 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:35.715504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c48434-7e3a-46fc-a534-e587e5c986f5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.723301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1310/2587] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:35.725862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7681e-e03b-4690-a825-7983f4c0dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.727111 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7681e-e03b-4690-a825-7983f4c0dd00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.727716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7681e-e03b-4690-a825-7983f4c0dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.728174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7681e-e03b-4690-a825-7983f4c0dd00 admin admin] PATH_INFO: `/v3/projects/fd429ab2fd1641c2b7260c59d1b3a9b1/users/a213e36d23f24c8dbc30cf2ceedc0c36/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.728783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cad3d6-b73a-45bb-bbdf-9049516a268b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.729078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cad3d6-b73a-45bb-bbdf-9049516a268b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.729361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cad3d6-b73a-45bb-bbdf-9049516a268b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.731700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7681e-e03b-4690-a825-7983f4c0dd00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd429ab2fd1641c2b7260c59d1b3a9b1, user_id=a213e36d23f24c8dbc30cf2ceedc0c36, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.734919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7681e-e03b-4690-a825-7983f4c0dd00 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.756873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1278/2588] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:35 2022] PUT /identity/v3/projects/fd429ab2fd1641c2b7260c59d1b3a9b1/users/a213e36d23f24c8dbc30cf2ceedc0c36/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:35.757092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e6cad3d6-b73a-45bb-bbdf-9049516a268b None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:35.757092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e6cad3d6-b73a-45bb-bbdf-9049516a268b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:35.759078 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.760336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.760627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.828939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] MFA Rules not processed for user `a213e36d23f24c8dbc30cf2ceedc0c36`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:35.830305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:35.845481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:35.896609 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1311/2589] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:35.899876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cad3d6-b73a-45bb-bbdf-9049516a268b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.916352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cad3d6-b73a-45bb-bbdf-9049516a268b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.929287 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.930206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.930500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.930782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.931453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1279/2590] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:35.931573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:35.931791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.933424 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:35.942066 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.944742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:35.956850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1312/2591] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:35.962628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:35.977406 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6dae9-2974-4313-9cd6-00b4f4bb0b8a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:35.982972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03a001-65d0-4b2a-ba22-dfc2a9a81959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:35.983613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03a001-65d0-4b2a-ba22-dfc2a9a81959 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:35.983896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03a001-65d0-4b2a-ba22-dfc2a9a81959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:35.984163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03a001-65d0-4b2a-ba22-dfc2a9a81959 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:35.984797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03a001-65d0-4b2a-ba22-dfc2a9a81959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:35.987001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03a001-65d0-4b2a-ba22-dfc2a9a81959 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.031317 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e41a1-7d5f-432f-9496-c05ac93ae2f3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.035151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1313/2592] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:35 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:36.036577 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a029d-ffce-4213-a4a9-bd02cea25da0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.036883 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a029d-ffce-4213-a4a9-bd02cea25da0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.037142 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a029d-ffce-4213-a4a9-bd02cea25da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.071005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ad5a029d-ffce-4213-a4a9-bd02cea25da0 None None] MFA Rules not processed for user `6d2c1bdf17fd485fb59273543850f2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:36.072372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ad5a029d-ffce-4213-a4a9-bd02cea25da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.098349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.099091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.099401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.099696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.147411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1314/2593] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:36.148349 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a029d-ffce-4213-a4a9-bd02cea25da0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.152408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:36.172761 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:36.175654 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.204355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1280/2594] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:36.215093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.219540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.262960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:36.281829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a029d-ffce-4213-a4a9-bd02cea25da0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.286992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.287659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.287942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.288215 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.288858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:36.290826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.317821 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1315/2595] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:36 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:36.318660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.322446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.339954 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab766f85-9d16-437d-98f6-1b60069139a3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.345801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.346441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.346706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905057f-1339-4438-b612-1f074b582d4d service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.347606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.347905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.349866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.355311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.355947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.356358 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.356699 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.358742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.441638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:36.443555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.453915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1281/2596] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:36.454808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.467329 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be8df5-bd42-4883-88b4-3586015818f2 service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.473910 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.474652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.475007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.476364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.477210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:36.479501 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.489006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:36.490617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.502360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1282/2597] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:36.504790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.505011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1316/2598] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:36.518513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3b31d-2f3a-4eb0-b6d9-32244afc75d3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.528703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fb7ab-5804-42d5-8aa5-0e962379c69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.528703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fb7ab-5804-42d5-8aa5-0e962379c69a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.529217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fb7ab-5804-42d5-8aa5-0e962379c69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.529889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fb7ab-5804-42d5-8aa5-0e962379c69a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.530886 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fb7ab-5804-42d5-8aa5-0e962379c69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:36.533082 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fb7ab-5804-42d5-8aa5-0e962379c69a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.581785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1283/2599] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:36.583671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.601671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7c250-7490-43b0-a8fb-bfadd4c659e1 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.607359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.608379 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.609021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.609470 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] PATH_INFO: `/v3/projects/862629108fd34417bed67e1b5eedbe92/users/cfc4077ebd9d4e05960fb68cc2406a87/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.639214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862629108fd34417bed67e1b5eedbe92, user_id=cfc4077ebd9d4e05960fb68cc2406a87, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:36.645660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:36.680272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1317/2600] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:36 2022] PUT /identity/v3/projects/862629108fd34417bed67e1b5eedbe92/users/cfc4077ebd9d4e05960fb68cc2406a87/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:36.680986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:36.681288 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:36.681575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:36.707919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:36.709667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.834119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.837557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.846145 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1284/2601] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:36.849641 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:36.852858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:36.942441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:36.994285 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f1ba1-0cc6-49d5-814c-75dad8120418 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.994433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe7247-a8c2-477b-ae34-2b5ba9307bb0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:36.999312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.999522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e2947-7733-475e-a1f8-1277e3ecb4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:36.999771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:37.000341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:37.001282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:37.004733 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e2947-7733-475e-a1f8-1277e3ecb4db service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:37.006833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e2947-7733-475e-a1f8-1277e3ecb4db service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:37.007131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e2947-7733-475e-a1f8-1277e3ecb4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:37.009668 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6f8e2947-7733-475e-a1f8-1277e3ecb4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:37.037122 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:37.038947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:37.047448 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:37.057708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1285/2602] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:37.133317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e2947-7733-475e-a1f8-1277e3ecb4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:37.139257 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e2947-7733-475e-a1f8-1277e3ecb4db service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:37.164027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1318/2603] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:38.168114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:38.180802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db414b79-f943-4319-be35-4558c33a3c17 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:38.187678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:38.188703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:38.189176 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:38.189921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:38.192476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:38.232319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:38.234069 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:38.265736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1286/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:38.496468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda83a5f-c7dc-4a8d-b21a-e1ff8e10cd70 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:38.497562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda83a5f-c7dc-4a8d-b21a-e1ff8e10cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:38.498014 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda83a5f-c7dc-4a8d-b21a-e1ff8e10cd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:38.532199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-eda83a5f-c7dc-4a8d-b21a-e1ff8e10cd70 None None] MFA Rules not processed for user `cfc4077ebd9d4e05960fb68cc2406a87`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:38.534353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-eda83a5f-c7dc-4a8d-b21a-e1ff8e10cd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:38.666094 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1319/2605] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:38.666379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:38.690889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7b63-ec12-4d2b-84cd-4c823f6712f8 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:38.698142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:38.699032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:38.699716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:38.700142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:38.700942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:38.703504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:38.748955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1287/2606] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:38 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:38.755045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda83a5f-c7dc-4a8d-b21a-e1ff8e10cd70 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:38.793230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda83a5f-c7dc-4a8d-b21a-e1ff8e10cd70 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:38.965270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:38.965944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:38.966283 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:38.966562 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:38.967238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:38.969514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:39.004943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1320/2607] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:38 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:39.008969 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:39.020862 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4db020-70be-486b-98ff-fbd3f3944f93 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:39.025985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:39.026647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:39.026937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:39.027237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:39.027943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:39.029634 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:39.042146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1288/2608] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:39.046654 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:39.060554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffec7-8b8e-4ae9-aeea-5b6172d0561b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:39.070212 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:39.071169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:39.071483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:39.071926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] PATH_INFO: `/v3/projects/59f74258d2804930a74fb18db98f4d92/users/80b53c816b034819a09e4acd18c598c3/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:39.093219 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f74258d2804930a74fb18db98f4d92, user_id=80b53c816b034819a09e4acd18c598c3, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:39.098898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:39.122198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1321/2609] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:39 2022] PUT /identity/v3/projects/59f74258d2804930a74fb18db98f4d92/users/80b53c816b034819a09e4acd18c598c3/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:39.122710 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:39.141328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe076be-92fe-46c5-aa76-f2ac068e339c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:39.231963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:39.236150 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:39.236700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:39.237007 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:39.237865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:39.243793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:39.258175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1289/2610] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:39.265423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:39.278126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb941d82-69b0-450a-b805-b85de6dbd63e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:39.293918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:39.294631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:39.294976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:39.295368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] PATH_INFO: `/v3/projects/59f74258d2804930a74fb18db98f4d92/users/80b53c816b034819a09e4acd18c598c3/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:39.297996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f74258d2804930a74fb18db98f4d92, user_id=80b53c816b034819a09e4acd18c598c3, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:39.301779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:39.315051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1322/2611] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:39 2022] PUT /identity/v3/projects/59f74258d2804930a74fb18db98f4d92/users/80b53c816b034819a09e4acd18c598c3/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:39.668937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:39.672468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:39.859447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909a88-b93c-4097-8458-03ebda17e2c3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:39.909959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:40.016541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f7acc-df67-4501-ad9f-b7362d46b67b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:40.017318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f7acc-df67-4501-ad9f-b7362d46b67b service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:40.017816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f7acc-df67-4501-ad9f-b7362d46b67b service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:40.018161 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f7acc-df67-4501-ad9f-b7362d46b67b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:40.027090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a94f7acc-df67-4501-ad9f-b7362d46b67b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:40.057616 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f7acc-df67-4501-ad9f-b7362d46b67b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:40.059411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f7acc-df67-4501-ad9f-b7362d46b67b service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:40.084235 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1290/2612] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:40.196801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:40.201834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:40.361618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c328a7-d2d7-4754-bcc3-944e27ebe475 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:40.379547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:40.382266 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:40.382506 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:40.382801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] PATH_INFO: `/v3/users/f38eb17e93c14f4da999c1df2440d41c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:40.387409 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38eb17e93c14f4da999c1df2440d41c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:40.389626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:40.503093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38eb17e93c14f4da999c1df2440d41c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:40.504258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38eb17e93c14f4da999c1df2440d41c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:40.508026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:40.508579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:40.508579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:40.514778 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38eb17e93c14f4da999c1df2440d41c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:40.515882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38eb17e93c14f4da999c1df2440d41c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:40.534299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e0570eed-e089-4b54-94c9-25f9ace07bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38eb17e93c14f4da999c1df2440d41c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:40.569604 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] MFA Rules not processed for user `80b53c816b034819a09e4acd18c598c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:40.570760 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:40.573539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1323/2613] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:40 2022] DELETE /identity/v3/users/f38eb17e93c14f4da999c1df2440d41c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:40.578810 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:40.579481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:40.580050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:40.618723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:40.621029 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:40.744921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1291/2614] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:40.762293 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1324/2615] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:40.764928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:40.771144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:40.778187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:40.780256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:40.847377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:40.908786 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad029750-363e-45f4-a85b-05b20f2b85db None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:40.932852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:40.933952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:40.934287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:40.935881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:40.938526 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:40.989457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fa635-ae9d-4298-8c7c-68c01f43e743 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:41.009992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:41.010645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:41.011435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:41.011702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:41.034397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:41.038376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:41.069203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1325/2616] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:41.090023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:41.138284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:41.140265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:41.155312 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1292/2617] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:42.654012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:42.656113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:42.683332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64b69-d835-414e-af63-2591aee6c4c6 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:42.697030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:42.698352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:42.698352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:42.698749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] PATH_INFO: `/v3/projects/32e331d852ea4fe397957b8e5a14fb06` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:42.700341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e331d852ea4fe397957b8e5a14fb06)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:42.703134 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:42.821709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e331d852ea4fe397957b8e5a14fb06', 'request_id': 'req-64972a61-991e-4b9e-8ce3-ed9954a02ff1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:42.835744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e331d852ea4fe397957b8e5a14fb06', 'request_id': 'req-64972a61-991e-4b9e-8ce3-ed9954a02ff1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:42.860130 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e331d852ea4fe397957b8e5a14fb06', 'request_id': 'req-64972a61-991e-4b9e-8ce3-ed9954a02ff1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:42.886002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] The token cache is being invalidate because project 32e331d852ea4fe397957b8e5a14fb06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:42.886433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:42.887558 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:42.890254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1326/2618] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:42 2022] DELETE /identity/v3/projects/32e331d852ea4fe397957b8e5a14fb06 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:42.894798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:42.912579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9272f6-1488-49e2-b7a0-24f3a0d09e40 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:43.087706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:43.095436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:43.095896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:43.096444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] PATH_INFO: `/v3/users/1daa433397f34fba96cbd988f54a57a8` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:43.097911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1daa433397f34fba96cbd988f54a57a8)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:43.101206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:43.146507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:43.151234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:43.232627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1daa433397f34fba96cbd988f54a57a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:43.248585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1daa433397f34fba96cbd988f54a57a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:43.258549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972a61-991e-4b9e-8ce3-ed9954a02ff1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:43.263438 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:43.264478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:43.264966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:43.265429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:43.269608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:43.280512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1daa433397f34fba96cbd988f54a57a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:43.281359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1daa433397f34fba96cbd988f54a57a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:43.282019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1daa433397f34fba96cbd988f54a57a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:43.349149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1293/2619] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:42 2022] DELETE /identity/v3/users/1daa433397f34fba96cbd988f54a57a8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:43.514143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:43.516253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:43.521881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:43.534359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1327/2620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:43.964204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:43.971005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:44.095028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1ecab-9e68-4129-ba66-451ae48f8684 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:44.106024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:44.107083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:44.107609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:44.108088 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] PATH_INFO: `/v3/projects/5a4704c500834cf095e982d7cc20cfd8` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:44.109775 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4704c500834cf095e982d7cc20cfd8)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:44.113887 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:44.131827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:44.185991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4704c500834cf095e982d7cc20cfd8', 'request_id': 'req-9d76fba5-c7a6-4075-9dec-47a085d4dded'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.194998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4704c500834cf095e982d7cc20cfd8', 'request_id': 'req-9d76fba5-c7a6-4075-9dec-47a085d4dded'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.202354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4704c500834cf095e982d7cc20cfd8', 'request_id': 'req-9d76fba5-c7a6-4075-9dec-47a085d4dded'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.216108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] The token cache is being invalidate because project 5a4704c500834cf095e982d7cc20cfd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:44.216366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.217490 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.219668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1294/2621] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:43 2022] DELETE /identity/v3/projects/5a4704c500834cf095e982d7cc20cfd8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:44.224357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:44.232039 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:44.250974 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a17d0-32c5-4a1a-990a-fea38c95b943 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:44.360860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:44.361589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:44.361875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:44.362152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] PATH_INFO: `/v3/users/a213e36d23f24c8dbc30cf2ceedc0c36` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:44.363648 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a213e36d23f24c8dbc30cf2ceedc0c36)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:44.365634 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:44.449843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a213e36d23f24c8dbc30cf2ceedc0c36'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.451214 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a213e36d23f24c8dbc30cf2ceedc0c36'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.464138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a213e36d23f24c8dbc30cf2ceedc0c36'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.465120 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a213e36d23f24c8dbc30cf2ceedc0c36'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.476725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a213e36d23f24c8dbc30cf2ceedc0c36'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.498253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1328/2622] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:44 2022] DELETE /identity/v3/users/a213e36d23f24c8dbc30cf2ceedc0c36 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:45.179054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.182567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.189510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.191494 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.299966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6057-1f4a-495b-a6f3-1ef0c0ba5b99 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.320490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.323619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.323913 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.324190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] PATH_INFO: `/v3/projects/fd429ab2fd1641c2b7260c59d1b3a9b1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.328057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd429ab2fd1641c2b7260c59d1b3a9b1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.330308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:45.332833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fba5-c7a6-4075-9dec-47a085d4dded admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.364919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:45.441617 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43d2a6-5f0d-42b4-9123-04c84c02b566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.442457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d2a6-5f0d-42b4-9123-04c84c02b566 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.442742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d2a6-5f0d-42b4-9123-04c84c02b566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.443016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d2a6-5f0d-42b4-9123-04c84c02b566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.445596 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4a43d2a6-5f0d-42b4-9123-04c84c02b566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.452769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd429ab2fd1641c2b7260c59d1b3a9b1', 'request_id': 'req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.465881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43d2a6-5f0d-42b4-9123-04c84c02b566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.466066 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd429ab2fd1641c2b7260c59d1b3a9b1', 'request_id': 'req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.468935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43d2a6-5f0d-42b4-9123-04c84c02b566 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:45.479520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd429ab2fd1641c2b7260c59d1b3a9b1', 'request_id': 'req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.491483 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1295/2623] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:45.497090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253b56a-b0a9-4863-af8c-c34c046f30bb None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.497649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253b56a-b0a9-4863-af8c-c34c046f30bb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.498286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253b56a-b0a9-4863-af8c-c34c046f30bb None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.500694 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1296/2624] 10.222.0.52 () {54 vars in 921 bytes} [Wed Aug 24 06:50:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:50:45.509929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] The token cache is being invalidate because project fd429ab2fd1641c2b7260c59d1b3a9b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:45.511519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.511519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-aa36b5ff-24e4-43ca-bf8e-74989e6f6948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.513307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1329/2625] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:45 2022] DELETE /identity/v3/projects/fd429ab2fd1641c2b7260c59d1b3a9b1 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:45.513732 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.514079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.514339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.522943 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.523347 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.523942 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.540429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] MFA Rules not processed for user `d115d3f3adc54a17bf80962a6ae3ab82`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:45.541742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.572876 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:45.574337 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.600177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:45.619166 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1297/2626] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Aug 24 06:50:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:45.620091 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.621592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.681540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1330/2627] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:45.681540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.681540 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.698879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e939bb-a658-4f80-a77c-df7dba895f05 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.702922 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb015c1d-c0cb-489b-b8ca-b2021bea80ef None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.705325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be4901-7dc5-48a3-b7f5-bc0cba2206b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.705946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4901-7dc5-48a3-b7f5-bc0cba2206b9 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.706260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4901-7dc5-48a3-b7f5-bc0cba2206b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.706478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4901-7dc5-48a3-b7f5-bc0cba2206b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.709795 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.710638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.710939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.711328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] PATH_INFO: `/v3/users/bfb5dc1fa16f465ba1ebca000b665d46` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.712947 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb5dc1fa16f465ba1ebca000b665d46)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.714973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:45.717877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be4901-7dc5-48a3-b7f5-bc0cba2206b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.719592 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be4901-7dc5-48a3-b7f5-bc0cba2206b9 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:45.734793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1331/2628] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:45.736245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10e03-8329-47dc-a3a4-35442f01ba06 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.736625 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10e03-8329-47dc-a3a4-35442f01ba06 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.737264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10e03-8329-47dc-a3a4-35442f01ba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.766901 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-10d10e03-8329-47dc-a3a4-35442f01ba06 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:45.769170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-10d10e03-8329-47dc-a3a4-35442f01ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.779873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1332/2629] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:45.787725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10e03-8329-47dc-a3a4-35442f01ba06 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.788283 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb5dc1fa16f465ba1ebca000b665d46'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.799291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10e03-8329-47dc-a3a4-35442f01ba06 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.801203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb5dc1fa16f465ba1ebca000b665d46'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.805082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.805933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.807517 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.807892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.808716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:45.809117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.811231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:45.821030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb5dc1fa16f465ba1ebca000b665d46'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.822087 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb5dc1fa16f465ba1ebca000b665d46'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.822809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb5dc1fa16f465ba1ebca000b665d46'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.824192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1333/2630] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:45.830431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.832947 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.848220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee238a5-a415-4b54-945e-997223ffae4d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.848976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1298/2631] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:45 2022] DELETE /identity/v3/users/bfb5dc1fa16f465ba1ebca000b665d46 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:45.922774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.923655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.924021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.924362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.925217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.927255 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:45.954421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:45.963368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1334/2632] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:45 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:45.967179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.979956 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2f8b5-5de5-443b-92df-33b5ff181ba7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.064829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.065448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.066028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.066319 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.067060 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.069275 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.071234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.079385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.092830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:46.098114 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cc834-42ef-4e02-85e7-3fecafc16435 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.100609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1299/2633] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:45 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:46.103568 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.107855 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.108508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.108862 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.109062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] PATH_INFO: `/v3/projects/0aec98515a95467aa9a43853bfdcb7d7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.110268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aec98515a95467aa9a43853bfdcb7d7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.112792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.119894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79e9f5-ee1b-4e70-92e9-6c3e68cfce26 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.126717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.127748 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.128249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.128592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.131362 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.135571 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.150460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1300/2634] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:46.152237 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.166231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658a0d1-ff73-447a-88a7-9056656c73fc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.171798 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.172484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.173025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.173447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.174454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.175890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.190868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aec98515a95467aa9a43853bfdcb7d7', 'request_id': 'req-2db4e2a1-582d-4c8d-941f-8cb9213e5370'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.199818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aec98515a95467aa9a43853bfdcb7d7', 'request_id': 'req-2db4e2a1-582d-4c8d-941f-8cb9213e5370'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.208200 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aec98515a95467aa9a43853bfdcb7d7', 'request_id': 'req-2db4e2a1-582d-4c8d-941f-8cb9213e5370'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.241874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] The token cache is being invalidate because project 0aec98515a95467aa9a43853bfdcb7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:46.243064 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.244243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2db4e2a1-582d-4c8d-941f-8cb9213e5370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.250282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1335/2635] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:46 2022] DELETE /identity/v3/projects/0aec98515a95467aa9a43853bfdcb7d7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:46.252930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae74c3-600b-4ece-9193-d8f7bb39bc6a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.253701 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae74c3-600b-4ece-9193-d8f7bb39bc6a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.254397 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae74c3-600b-4ece-9193-d8f7bb39bc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.280789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d2ae74c3-600b-4ece-9193-d8f7bb39bc6a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:46.282710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d2ae74c3-600b-4ece-9193-d8f7bb39bc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.349642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1301/2636] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:46.350686 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.353201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.376227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761fb6c-737e-4a73-b51f-ed1ac4bfeea8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.384457 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1336/2637] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:46.393696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4eea-7b20-4ff0-97ae-aeda9609a91f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.394057 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4eea-7b20-4ff0-97ae-aeda9609a91f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.394408 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4eea-7b20-4ff0-97ae-aeda9609a91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.418524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2d2d4eea-7b20-4ff0-97ae-aeda9609a91f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:46.420096 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2d2d4eea-7b20-4ff0-97ae-aeda9609a91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.431080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1337/2638] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:46.435784 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4eea-7b20-4ff0-97ae-aeda9609a91f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.447872 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d4eea-7b20-4ff0-97ae-aeda9609a91f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.455727 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.456432 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.456782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.457126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.458496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:46.458878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.461486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.480138 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db102ad8-3904-4d52-bae6-65b3833d66bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.481453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db102ad8-3904-4d52-bae6-65b3833d66bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.481792 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db102ad8-3904-4d52-bae6-65b3833d66bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.482075 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db102ad8-3904-4d52-bae6-65b3833d66bc admin admin] PATH_INFO: `/v3/projects/b54e396f5d804768af420949e4011de9/users/c4123cf83f4347beb113875f31679215/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.487252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1338/2639] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:46.488146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.498523 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02194c7b-9610-4932-b9cf-0707f57e4047 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.507152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.508238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.508624 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.508998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.509761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.511739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db102ad8-3904-4d52-bae6-65b3833d66bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54e396f5d804768af420949e4011de9, user_id=c4123cf83f4347beb113875f31679215, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.511874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.514733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db102ad8-3904-4d52-bae6-65b3833d66bc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.538836 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1302/2640] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:46 2022] PUT /identity/v3/projects/b54e396f5d804768af420949e4011de9/users/c4123cf83f4347beb113875f31679215/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:46.552728 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae2c7b-afe3-480f-ae3f-985a1887e4d1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.553296 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae2c7b-afe3-480f-ae3f-985a1887e4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.554255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae2c7b-afe3-480f-ae3f-985a1887e4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.555150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1339/2641] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:46.556065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.623371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-12ae2c7b-afe3-480f-ae3f-985a1887e4d1 None None] MFA Rules not processed for user `c4123cf83f4347beb113875f31679215`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:46.624721 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-12ae2c7b-afe3-480f-ae3f-985a1887e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.663740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:46.693911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042187-9d56-4e60-9cb0-2a2419af10ab admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.716431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.717081 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.717368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.717764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.719497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.720554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.752775 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1340/2642] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:46.757848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.771244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1303/2643] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:46.775612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae2c7b-afe3-480f-ae3f-985a1887e4d1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.777143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ccf2d-c885-40d9-aa00-c3ae61ed85e6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.783148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.783896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.784413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.784793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.785581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.787592 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.802868 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1341/2644] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:46.804262 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.809986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae2c7b-afe3-480f-ae3f-985a1887e4d1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.816319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cf724-a557-475e-9389-a6d53f46a9f2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.819949 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.820779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.823999 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.824371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.824657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.825001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.825480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.826032 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.826395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.828169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.829708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.830543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.850247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1342/2645] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:46.853795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.875419 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd2cc0-2e7a-476b-a96c-573f5686f93b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.880404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1304/2646] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:46.881419 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.920054 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:46.958465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8dbe0-b453-4ae6-b15f-b7ca939f1a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.959151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8dbe0-b453-4ae6-b15f-b7ca939f1a49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.959471 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8dbe0-b453-4ae6-b15f-b7ca939f1a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.959753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8dbe0-b453-4ae6-b15f-b7ca939f1a49 admin admin] PATH_INFO: `/v3/projects/0bc596aa9feb471a9e0d32cb4a951002/users/66971d1a42114fc7acd7f8f372ea681e/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.959913 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74038b9-0833-4b64-a257-9f12adf64acd admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.966601 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.967507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.967655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.967825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.968465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.971361 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.978773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8dbe0-b453-4ae6-b15f-b7ca939f1a49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc596aa9feb471a9e0d32cb4a951002, user_id=66971d1a42114fc7acd7f8f372ea681e, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.981874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8dbe0-b453-4ae6-b15f-b7ca939f1a49 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.001174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1305/2647] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:47.001395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1343/2648] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:46 2022] PUT /identity/v3/projects/0bc596aa9feb471a9e0d32cb4a951002/users/66971d1a42114fc7acd7f8f372ea681e/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:47.003049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.011801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.012206 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.012493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.018102 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81101fb-afae-4972-84b3-a7552a48c957 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.041078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] MFA Rules not processed for user `66971d1a42114fc7acd7f8f372ea681e`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:47.042367 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:47.091733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.092379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.092715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.092998 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.093667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.096004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.109982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1306/2649] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:47.114606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.133315 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4475-767c-4f86-91a7-9a3b39fd4a2d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.147229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.147229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.147229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.147229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] PATH_INFO: `/v3/projects/90f7baeae70a48caaa460359e9a3ac54/users/9b40a622d1a64d55ac355a2507b48698/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.148449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1344/2650] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:47.158198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f7baeae70a48caaa460359e9a3ac54, user_id=9b40a622d1a64d55ac355a2507b48698, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.161445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.164252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.166848 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:47.180535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1307/2651] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:47 2022] PUT /identity/v3/projects/90f7baeae70a48caaa460359e9a3ac54/users/9b40a622d1a64d55ac355a2507b48698/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:47.184622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.199267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:47.259689 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae37b7f-815b-4dfd-a131-7329e21da293 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.261484 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9cdd1-d3d6-4b65-a03e-8da732c8db2f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.265053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.265670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.265954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.266228 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.266883 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.268681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.281529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1308/2652] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:47.286458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.306904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2b63d-95a2-4fb7-9d33-d5a9baf642d7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.312597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19946d-0e0d-455b-a762-a5cabe3e04b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.313269 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19946d-0e0d-455b-a762-a5cabe3e04b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.313608 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19946d-0e0d-455b-a762-a5cabe3e04b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.314020 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19946d-0e0d-455b-a762-a5cabe3e04b4 admin admin] PATH_INFO: `/v3/projects/90f7baeae70a48caaa460359e9a3ac54/users/9b40a622d1a64d55ac355a2507b48698/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.316155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19946d-0e0d-455b-a762-a5cabe3e04b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f7baeae70a48caaa460359e9a3ac54, user_id=9b40a622d1a64d55ac355a2507b48698, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.318789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19946d-0e0d-455b-a762-a5cabe3e04b4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.334672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.335667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.336052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.336495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.339446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1309/2653] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:47 2022] PUT /identity/v3/projects/90f7baeae70a48caaa460359e9a3ac54/users/9b40a622d1a64d55ac355a2507b48698/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:47.342848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.343468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.344282 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.384022 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:47.387651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] MFA Rules not processed for user `9b40a622d1a64d55ac355a2507b48698`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:47.389045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:47.439061 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.441716 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.452803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1345/2654] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:47.480293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1310/2655] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:47.499638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.501102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:47.603818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327ec7e-0221-4e28-ac96-b4df358088b2 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.617638 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:47.686708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.687663 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.688010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.688377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.769612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.772447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.800991 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1346/2656] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:52.634573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.636579 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:52.724937 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc84128-609c-4230-9a75-0a69f54f0d23 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.731646 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.732434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.732847 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.733134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] PATH_INFO: `/v3/users/7957b80bf6e843d4a7a0b11f13a80db8` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.734277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7957b80bf6e843d4a7a0b11f13a80db8)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.736827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.793743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7957b80bf6e843d4a7a0b11f13a80db8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:52.800824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7957b80bf6e843d4a7a0b11f13a80db8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:52.811629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7957b80bf6e843d4a7a0b11f13a80db8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:52.812341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7957b80bf6e843d4a7a0b11f13a80db8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:52.812983 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7957b80bf6e843d4a7a0b11f13a80db8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:52.831170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1311/2657] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:52 2022] DELETE /identity/v3/users/7957b80bf6e843d4a7a0b11f13a80db8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:53.020899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:53.039904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9e16b-651a-45ff-8cbf-7e9986d9aea6 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:53.114143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:53.114803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:53.115090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:53.115403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] PATH_INFO: `/v3/projects/0bf38c354bea44f5a503e76fa53b20f6` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:53.116543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf38c354bea44f5a503e76fa53b20f6)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:53.118504 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:53.170886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf38c354bea44f5a503e76fa53b20f6', 'request_id': 'req-643be838-6f57-4f50-b3a4-7d4256ca5a6f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.178135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf38c354bea44f5a503e76fa53b20f6', 'request_id': 'req-643be838-6f57-4f50-b3a4-7d4256ca5a6f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.184423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf38c354bea44f5a503e76fa53b20f6', 'request_id': 'req-643be838-6f57-4f50-b3a4-7d4256ca5a6f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.195977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:53.202898 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] The token cache is being invalidate because project 0bf38c354bea44f5a503e76fa53b20f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:53.203396 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.204282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.206357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1347/2658] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:53 2022] DELETE /identity/v3/projects/0bf38c354bea44f5a503e76fa53b20f6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:53.210931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:53.212810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:53.227963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cf529-318c-49fc-93e2-07f6089e65bf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:53.240868 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:53.291945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:53.292613 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:53.292953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:53.293289 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] PATH_INFO: `/v3/users/bd5127b5d0034af2a4bac2e0330a400c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:53.294665 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd5127b5d0034af2a4bac2e0330a400c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:53.296815 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:53.359293 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd5127b5d0034af2a4bac2e0330a400c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.367919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd5127b5d0034af2a4bac2e0330a400c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.377387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd5127b5d0034af2a4bac2e0330a400c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.378438 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd5127b5d0034af2a4bac2e0330a400c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.379220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd5127b5d0034af2a4bac2e0330a400c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.400236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1312/2659] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:53 2022] DELETE /identity/v3/users/bd5127b5d0034af2a4bac2e0330a400c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:53.585361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:53.587003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:53.601718 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643be838-6f57-4f50-b3a4-7d4256ca5a6f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:53.661388 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:53.661962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:53.662240 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:53.662513 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] PATH_INFO: `/v3/projects/3a06b7cea7d34413b2f95c1db2583af0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:53.663676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a06b7cea7d34413b2f95c1db2583af0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:53.665866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:53.717449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a06b7cea7d34413b2f95c1db2583af0', 'request_id': 'req-917ae60b-3121-4b43-b337-a7fa5c693d73'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.724795 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a06b7cea7d34413b2f95c1db2583af0', 'request_id': 'req-917ae60b-3121-4b43-b337-a7fa5c693d73'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.731045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a06b7cea7d34413b2f95c1db2583af0', 'request_id': 'req-917ae60b-3121-4b43-b337-a7fa5c693d73'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.745614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] The token cache is being invalidate because project 3a06b7cea7d34413b2f95c1db2583af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:53.745926 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.746641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:53.761442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1348/2660] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:53 2022] DELETE /identity/v3/projects/3a06b7cea7d34413b2f95c1db2583af0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:54.517028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:54.518636 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:54.580895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc68941-d9cd-44d2-84c8-519b8439988e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:54.633530 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f9129-616a-45f2-8e1b-381630692f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:54.634151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f9129-616a-45f2-8e1b-381630692f19 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:54.634430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f9129-616a-45f2-8e1b-381630692f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:54.634700 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f9129-616a-45f2-8e1b-381630692f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:54.636546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b20f9129-616a-45f2-8e1b-381630692f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:54.642678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:54.705434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f9129-616a-45f2-8e1b-381630692f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:54.706839 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f9129-616a-45f2-8e1b-381630692f19 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:54.716647 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1313/2661] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:55.163328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:55.163328 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:55.232314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ae60b-3121-4b43-b337-a7fa5c693d73 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:55.237330 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:55.237964 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:55.238245 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:55.238519 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] PATH_INFO: `/v3/users/66971d1a42114fc7acd7f8f372ea681e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:55.239695 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66971d1a42114fc7acd7f8f372ea681e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:55.241610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:55.278362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:55.293463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66971d1a42114fc7acd7f8f372ea681e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.294237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66971d1a42114fc7acd7f8f372ea681e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.302421 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66971d1a42114fc7acd7f8f372ea681e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.303145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66971d1a42114fc7acd7f8f372ea681e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.313027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66971d1a42114fc7acd7f8f372ea681e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.334140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1349/2662] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:55 2022] DELETE /identity/v3/users/66971d1a42114fc7acd7f8f372ea681e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:55.338837 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:55.339254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:55.339635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:55.360832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:55.362291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:55.430813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1314/2663] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:55.441843 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:55.446567 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:55.513524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfa76f-b9c6-4ef1-bc86-61bd28e748c6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:55.515649 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00527b-6a10-4903-b59d-503690eac799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:55.516264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00527b-6a10-4903-b59d-503690eac799 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:55.516550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00527b-6a10-4903-b59d-503690eac799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:55.516822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00527b-6a10-4903-b59d-503690eac799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:55.528075 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00527b-6a10-4903-b59d-503690eac799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:55.529530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00527b-6a10-4903-b59d-503690eac799 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:55.540723 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1350/2664] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:55.838454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:55.840233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:55.856812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f0ebe-e2f3-43bb-953f-4b39b5be6505 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:55.862410 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:55.862797 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:55.863134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:55.863504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] PATH_INFO: `/v3/projects/0bc596aa9feb471a9e0d32cb4a951002` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:55.864727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc596aa9feb471a9e0d32cb4a951002)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:55.866944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:55.922046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc596aa9feb471a9e0d32cb4a951002', 'request_id': 'req-9d25d3e8-4499-43ea-992f-039a81056b58'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.932264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc596aa9feb471a9e0d32cb4a951002', 'request_id': 'req-9d25d3e8-4499-43ea-992f-039a81056b58'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.942384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc596aa9feb471a9e0d32cb4a951002', 'request_id': 'req-9d25d3e8-4499-43ea-992f-039a81056b58'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.961960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] The token cache is being invalidate because project 0bc596aa9feb471a9e0d32cb4a951002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:55.962252 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.962909 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d25d3e8-4499-43ea-992f-039a81056b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.964870 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1315/2665] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:55 2022] DELETE /identity/v3/projects/0bc596aa9feb471a9e0d32cb4a951002 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:55.969882 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd216d5f-6b37-4694-b32c-8a9033bf8a45 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:55.970176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd216d5f-6b37-4694-b32c-8a9033bf8a45 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:55.970456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd216d5f-6b37-4694-b32c-8a9033bf8a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:55.991162 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fd216d5f-6b37-4694-b32c-8a9033bf8a45 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:55.992415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fd216d5f-6b37-4694-b32c-8a9033bf8a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:56.062488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1351/2666] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:56.070602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c00fa-c45d-4add-a4cf-8d9c96bd9e1d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.070878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c00fa-c45d-4add-a4cf-8d9c96bd9e1d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.071184 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c00fa-c45d-4add-a4cf-8d9c96bd9e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.092959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ea7c00fa-c45d-4add-a4cf-8d9c96bd9e1d None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:56.094223 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ea7c00fa-c45d-4add-a4cf-8d9c96bd9e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:56.103756 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1316/2667] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:56.108714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd216d5f-6b37-4694-b32c-8a9033bf8a45 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.124575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd216d5f-6b37-4694-b32c-8a9033bf8a45 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.131161 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.131808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.132089 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.132365 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.133052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:56.133358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.134992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.148297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1352/2668] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:56.152737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c00fa-c45d-4add-a4cf-8d9c96bd9e1d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.162569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c00fa-c45d-4add-a4cf-8d9c96bd9e1d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.168034 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.168706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.169045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.169380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.170058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.172190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.196357 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:56.210449 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1317/2669] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:56.216315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.226646 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad286d-3f0e-44ff-bfb1-b964b7fa3011 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.264666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:56.294541 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.295422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.295776 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.296173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.296770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.299035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.326831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1353/2670] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:56.328149 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.338898 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9faa7-2c35-4dc5-82d3-83179e491c8d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.344027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.344670 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.344984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.345221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.346029 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.348093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.363231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1318/2671] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:56.365191 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.375997 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f25874-1a7c-423c-9a19-71c7e18e148f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.381985 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cf48c-282c-49af-8294-4c7c602a90b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.382619 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf48c-282c-49af-8294-4c7c602a90b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.382902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf48c-282c-49af-8294-4c7c602a90b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.383178 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf48c-282c-49af-8294-4c7c602a90b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.383874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cf48c-282c-49af-8294-4c7c602a90b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.385520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cf48c-282c-49af-8294-4c7c602a90b6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.396697 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1354/2672] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:56.401181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.411752 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746ad71-36ad-485f-a343-bd98bdc06790 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.417254 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.417920 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.418203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.418474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] PATH_INFO: `/v3/projects/34069499b9f64b81bcaa49dfc64ff1cf/users/f8d3910de1e9444ab2e603bfe7d045fb/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.433928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34069499b9f64b81bcaa49dfc64ff1cf, user_id=f8d3910de1e9444ab2e603bfe7d045fb, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.436952 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.450788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1319/2673] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:56 2022] PUT /identity/v3/projects/34069499b9f64b81bcaa49dfc64ff1cf/users/f8d3910de1e9444ab2e603bfe7d045fb/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:56.462935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc48e1-021b-417b-a851-1c2251f22ec9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.463260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc48e1-021b-417b-a851-1c2251f22ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.463539 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc48e1-021b-417b-a851-1c2251f22ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.499737 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7bdc48e1-021b-417b-a851-1c2251f22ec9 None None] MFA Rules not processed for user `f8d3910de1e9444ab2e603bfe7d045fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:56.501290 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7bdc48e1-021b-417b-a851-1c2251f22ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:56.583545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1355/2674] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:56.588244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.663305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bd3a2-d7a1-4c7c-b9c1-b96eb28ccd23 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.668206 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.668822 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.669107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.669385 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.670049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.672139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.700356 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1320/2675] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:56.704565 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc48e1-021b-417b-a851-1c2251f22ec9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.716148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc48e1-021b-417b-a851-1c2251f22ec9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.788294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.789021 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.789310 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.789591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.790335 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.792231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.818216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1356/2676] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:56.822225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.833841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483b199-e1c7-4671-92c0-c422e46e75e9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.839645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.840389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.840749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.841058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.841977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.844754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.856350 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1321/2677] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:56.861393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.872296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e955e4a-ecbc-482c-9ce7-7b6450b4f846 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.880234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.880910 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.881189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.881464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] PATH_INFO: `/v3/projects/f7ab17292dd84a4dbf1391d82e01bd2a/users/64e1b88a25f6455f976b932109112f89/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.903632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ab17292dd84a4dbf1391d82e01bd2a, user_id=64e1b88a25f6455f976b932109112f89, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.907645 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.921870 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1357/2678] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:56 2022] PUT /identity/v3/projects/f7ab17292dd84a4dbf1391d82e01bd2a/users/64e1b88a25f6455f976b932109112f89/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:56.926155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.936639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9589b2d-731f-47fe-8339-8f2191cc4998 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:57.002620 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7afd6-1b1a-4200-b9eb-885977678615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:57.003344 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7afd6-1b1a-4200-b9eb-885977678615 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.003629 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7afd6-1b1a-4200-b9eb-885977678615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.003903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7afd6-1b1a-4200-b9eb-885977678615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.004551 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7afd6-1b1a-4200-b9eb-885977678615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:57.006180 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7afd6-1b1a-4200-b9eb-885977678615 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:57.016672 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1322/2679] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:57.021280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:57.032535 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffc337-a670-4a99-b722-00b2036fa4b4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:57.039614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:57.040279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.040572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.040847 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] PATH_INFO: `/v3/projects/f7ab17292dd84a4dbf1391d82e01bd2a/users/64e1b88a25f6455f976b932109112f89/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.043065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ab17292dd84a4dbf1391d82e01bd2a, user_id=64e1b88a25f6455f976b932109112f89, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:57.045813 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:57.052222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:57.059305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1358/2680] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:57 2022] PUT /identity/v3/projects/f7ab17292dd84a4dbf1391d82e01bd2a/users/64e1b88a25f6455f976b932109112f89/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:57.070388 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70b53-b6d6-4e98-adb3-04182103f52e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.070681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70b53-b6d6-4e98-adb3-04182103f52e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.070985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70b53-b6d6-4e98-adb3-04182103f52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.100841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2cc70b53-b6d6-4e98-adb3-04182103f52e None None] MFA Rules not processed for user `64e1b88a25f6455f976b932109112f89`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:57.101981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2cc70b53-b6d6-4e98-adb3-04182103f52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:57.122549 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:57.176517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1323/2681] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:57.188824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:57.191281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:57.251183 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cfea2-e2eb-44c5-aafe-d02778a8b446 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:57.255657 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d95058-83a5-486c-b4c6-cac115767d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:57.256265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d95058-83a5-486c-b4c6-cac115767d4c service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.256544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d95058-83a5-486c-b4c6-cac115767d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.256817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d95058-83a5-486c-b4c6-cac115767d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.265832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d95058-83a5-486c-b4c6-cac115767d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:57.267297 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d95058-83a5-486c-b4c6-cac115767d4c service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:57.276981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1359/2682] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:57.707174 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc70b53-b6d6-4e98-adb3-04182103f52e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:57.774341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc70b53-b6d6-4e98-adb3-04182103f52e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:57.779103 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:57.779790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.780189 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.780470 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] PATH_INFO: `/v3/users/f8d3910de1e9444ab2e603bfe7d045fb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.781757 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d3910de1e9444ab2e603bfe7d045fb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:57.783991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:57.834606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d3910de1e9444ab2e603bfe7d045fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:57.841649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d3910de1e9444ab2e603bfe7d045fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:57.850719 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d3910de1e9444ab2e603bfe7d045fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:57.851456 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d3910de1e9444ab2e603bfe7d045fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:57.852157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d2455222-017f-4f31-a9a8-ec106c544969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d3910de1e9444ab2e603bfe7d045fb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:57.870124 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1324/2683] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:57 2022] DELETE /identity/v3/users/f8d3910de1e9444ab2e603bfe7d045fb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:57.874761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.875143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.875538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.877677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b67cc-f121-40e8-b8de-170a14e85361 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.877966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b67cc-f121-40e8-b8de-170a14e85361 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.878244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b67cc-f121-40e8-b8de-170a14e85361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.895704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:57.897027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:57.897638 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ce4b67cc-f121-40e8-b8de-170a14e85361 None None] MFA Rules not processed for user `0b23553f7543446cb5eca18f26e25816`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:57.898813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ce4b67cc-f121-40e8-b8de-170a14e85361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:57.937739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:57.961917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1360/2684] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:57.971341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1325/2685] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:57.973696 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:57.975093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:58.039536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728df5-fbf6-4965-a79f-b9bf832e1345 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:58.043995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:58.043995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:58.044378 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:58.044720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:58.056478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:58.058291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:58.074116 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1361/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:58.427958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b67cc-f121-40e8-b8de-170a14e85361 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:58.440199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b67cc-f121-40e8-b8de-170a14e85361 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:58.444659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:58.445277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:58.445715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:58.445994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:58.448079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:58.461593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:58.463175 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:58.473272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1326/2687] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:58.554833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:58.560584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:58.625814 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09e568-65dc-4ab3-8966-17d0ad0f46ac service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:58.639767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:58.640735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:58.641024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:58.641303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] PATH_INFO: `/v3/projects/34069499b9f64b81bcaa49dfc64ff1cf` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:58.642480 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=34069499b9f64b81bcaa49dfc64ff1cf)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:58.645402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:58.706962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:58.714153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34069499b9f64b81bcaa49dfc64ff1cf', 'request_id': 'req-28027a9f-b877-4f7e-91fd-7c1b98009ace'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.722914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34069499b9f64b81bcaa49dfc64ff1cf', 'request_id': 'req-28027a9f-b877-4f7e-91fd-7c1b98009ace'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.729955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34069499b9f64b81bcaa49dfc64ff1cf', 'request_id': 'req-28027a9f-b877-4f7e-91fd-7c1b98009ace'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.744953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] The token cache is being invalidate because project 34069499b9f64b81bcaa49dfc64ff1cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:58.745252 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.745897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.747461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1362/2688] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:58 2022] DELETE /identity/v3/projects/34069499b9f64b81bcaa49dfc64ff1cf => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:58.752199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:58.766316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592b1f-bb68-45df-a5e6-539b079027c4 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:58.840072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:58.840746 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:58.841253 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:58.841529 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] PATH_INFO: `/v3/users/64e1b88a25f6455f976b932109112f89` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:58.842745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e1b88a25f6455f976b932109112f89)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:58.845214 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:58.901425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e1b88a25f6455f976b932109112f89'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.909789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e1b88a25f6455f976b932109112f89'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.918198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e1b88a25f6455f976b932109112f89'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.918875 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e1b88a25f6455f976b932109112f89'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.919517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f7e21bea-dc55-4309-a932-25e447efb8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e1b88a25f6455f976b932109112f89'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:58.938005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1327/2689] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:58 2022] DELETE /identity/v3/users/64e1b88a25f6455f976b932109112f89 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:59.259592 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.261179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:59.275747 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28027a9f-b877-4f7e-91fd-7c1b98009ace admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.340474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.341316 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.341653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.341946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] PATH_INFO: `/v3/projects/f7ab17292dd84a4dbf1391d82e01bd2a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.343136 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ab17292dd84a4dbf1391d82e01bd2a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.345148 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.393186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ab17292dd84a4dbf1391d82e01bd2a', 'request_id': 'req-0542983a-851e-408f-bd4d-e9b22be7cace'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.399437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ab17292dd84a4dbf1391d82e01bd2a', 'request_id': 'req-0542983a-851e-408f-bd4d-e9b22be7cace'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.409509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ab17292dd84a4dbf1391d82e01bd2a', 'request_id': 'req-0542983a-851e-408f-bd4d-e9b22be7cace'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.424978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] The token cache is being invalidate because project f7ab17292dd84a4dbf1391d82e01bd2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:59.425507 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.426456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0542983a-851e-408f-bd4d-e9b22be7cace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.428159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1363/2690] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:59 2022] DELETE /identity/v3/projects/f7ab17292dd84a4dbf1391d82e01bd2a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:59.433428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab8d5-51f5-404a-a6f8-6cc58b0283b3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.433723 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab8d5-51f5-404a-a6f8-6cc58b0283b3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.433996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab8d5-51f5-404a-a6f8-6cc58b0283b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.454931 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9b6ab8d5-51f5-404a-a6f8-6cc58b0283b3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:59.456809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9b6ab8d5-51f5-404a-a6f8-6cc58b0283b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:59.503476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:59.520430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1328/2691] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:59.528307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24039c-60af-4c9a-8300-c17f2c79ceb0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.528600 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24039c-60af-4c9a-8300-c17f2c79ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.528873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24039c-60af-4c9a-8300-c17f2c79ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.558352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3b24039c-60af-4c9a-8300-c17f2c79ceb0 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:59.559608 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3b24039c-60af-4c9a-8300-c17f2c79ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:59.571828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1364/2692] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:59.573049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ab8d5-51f5-404a-a6f8-6cc58b0283b3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.585448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ab8d5-51f5-404a-a6f8-6cc58b0283b3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.593379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.594305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.594758 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.595170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.596160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:59.596507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.598159 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.615062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1329/2693] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:59.617070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24039c-60af-4c9a-8300-c17f2c79ceb0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.627443 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24039c-60af-4c9a-8300-c17f2c79ceb0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.632935 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.633545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.633826 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.634086 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.634719 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.636933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.670308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1365/2694] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:59.675597 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.686751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e07b32-0016-4ab3-95c9-75b40a313325 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.717265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.719322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:59.758363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:59.760535 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.761309 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.761671 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.762260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.762785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.765430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.792100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1330/2695] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:59.796160 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.802353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04cbfbc-2e4e-4063-aa72-c20f1e8212fa admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.806982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6537ca7-86d3-4817-ad77-76dc2745fb72 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.807769 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.808623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.808899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.809171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] PATH_INFO: `/v3/users/c4123cf83f4347beb113875f31679215` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.810658 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4123cf83f4347beb113875f31679215)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.812213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.812621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.812857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.813146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.813425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.814090 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.815771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.827162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1331/2696] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:59.831394 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.842110 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24a679-ecd4-4b0b-a0dd-03cdc335d0a4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.848229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.849249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.849600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.850168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.851451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.853270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.864284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1332/2697] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:59.868542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.879249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4123cf83f4347beb113875f31679215'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.880028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4123cf83f4347beb113875f31679215'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.880349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b4434-1b5b-4a0b-ad1b-4626a34ee064 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.887417 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2169af9-e83e-4c7d-9ac4-2ba9a162079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.888083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2169af9-e83e-4c7d-9ac4-2ba9a162079d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.888363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2169af9-e83e-4c7d-9ac4-2ba9a162079d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.888637 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2169af9-e83e-4c7d-9ac4-2ba9a162079d admin admin] PATH_INFO: `/v3/projects/eaae9976808d4eb994cddc6251948c2e/users/3e9b2d5386a444ed93c35708cd7b5666/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.891783 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4123cf83f4347beb113875f31679215'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.893253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4123cf83f4347beb113875f31679215'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.905578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b697b744-507d-4875-95eb-57ad55619cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4123cf83f4347beb113875f31679215'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:59.908373 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2169af9-e83e-4c7d-9ac4-2ba9a162079d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaae9976808d4eb994cddc6251948c2e, user_id=3e9b2d5386a444ed93c35708cd7b5666, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.913915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2169af9-e83e-4c7d-9ac4-2ba9a162079d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.931093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1366/2698] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:59 2022] DELETE /identity/v3/users/c4123cf83f4347beb113875f31679215 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:59.933036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1333/2699] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:59 2022] PUT /identity/v3/projects/eaae9976808d4eb994cddc6251948c2e/users/3e9b2d5386a444ed93c35708cd7b5666/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:59.943520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.943520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.943520 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.947446 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.950079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.950450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.962007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:59.963363 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:00.000696 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] MFA Rules not processed for user `3e9b2d5386a444ed93c35708cd7b5666`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:00.001926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:00.037425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1367/2700] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:00.054222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.062944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:00.092508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1334/2701] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:00.097285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.100048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:00.117618 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb59e5c-cde3-42c8-8d68-1092dec7110f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.125125 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.125981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.126476 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.126980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.127911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.131765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.139133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:00.176148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1335/2702] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:00 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:00.176966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.186886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:00.193557 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72c4a1-30e4-4492-8307-a8674fb41f13 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.301893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.302631 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.302918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.303220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.306437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680cda4-7b90-4635-a057-f0a2cb7cbc09 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.313005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.314944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.315315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.315612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.316057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.316869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.316994 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.319113 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.328896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1368/2703] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:00.345079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1336/2704] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:00 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:00.349959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.364043 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab7a7d-6393-4a2b-bf13-e8f7ca79027b service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.369440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.370072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.370357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.370636 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.371333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.372973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.384435 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1369/2705] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:00.388907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.401299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d552ec-5249-4efe-9788-76724ed4d533 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.407593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.408851 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.409244 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.409592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] PATH_INFO: `/v3/projects/3d069f5657a44b1eaa17f2cebc6c835b/users/b6c6d0bcb55740d59e5e61713d3ccfdf/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.434325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d069f5657a44b1eaa17f2cebc6c835b, user_id=b6c6d0bcb55740d59e5e61713d3ccfdf, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.437552 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.459656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1337/2706] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:00 2022] PUT /identity/v3/projects/3d069f5657a44b1eaa17f2cebc6c835b/users/b6c6d0bcb55740d59e5e61713d3ccfdf/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:00.464087 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.474797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3dfa8-efdd-462f-901c-8c9032c94f58 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.570189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac3826-e9ac-4d4d-8813-248968d36143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.572434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3826-e9ac-4d4d-8813-248968d36143 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.572902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3826-e9ac-4d4d-8813-248968d36143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.573352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3826-e9ac-4d4d-8813-248968d36143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.574777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac3826-e9ac-4d4d-8813-248968d36143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.577027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac3826-e9ac-4d4d-8813-248968d36143 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.593111 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1370/2707] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:00.598014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.615028 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cf9cc-3e35-42ad-80ff-5884e661798d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.621698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.622462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.623354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.623640 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] PATH_INFO: `/v3/projects/3d069f5657a44b1eaa17f2cebc6c835b/users/b6c6d0bcb55740d59e5e61713d3ccfdf/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.634658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d069f5657a44b1eaa17f2cebc6c835b, user_id=b6c6d0bcb55740d59e5e61713d3ccfdf, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.640965 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.661587 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1338/2708] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:00 2022] PUT /identity/v3/projects/3d069f5657a44b1eaa17f2cebc6c835b/users/b6c6d0bcb55740d59e5e61713d3ccfdf/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:00.676159 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.676640 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.677221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.712704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] MFA Rules not processed for user `b6c6d0bcb55740d59e5e61713d3ccfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:00.714613 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:00.804768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1371/2709] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:00.819240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:00.822408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:00.900986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bb9d1-c3eb-4f2a-85f9-3f84f12fec31 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:00.906172 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:00.906903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:00.907455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:00.907591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:00.975053 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:00.978584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:00.980768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:01.005330 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1339/2710] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:01.028380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:01.029549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:01.070307 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:01.172196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b635b8-a8fb-4b72-be10-4b2e64a1346b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:01.181984 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:01.182647 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:01.183111 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:01.183423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] PATH_INFO: `/v3/projects/b54e396f5d804768af420949e4011de9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:01.185543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54e396f5d804768af420949e4011de9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:01.188221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:01.262962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54e396f5d804768af420949e4011de9', 'request_id': 'req-4846b6dc-2cce-4e5b-8d52-925fcec8b461'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.278336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54e396f5d804768af420949e4011de9', 'request_id': 'req-4846b6dc-2cce-4e5b-8d52-925fcec8b461'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.290427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54e396f5d804768af420949e4011de9', 'request_id': 'req-4846b6dc-2cce-4e5b-8d52-925fcec8b461'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.322510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] The token cache is being invalidate because project b54e396f5d804768af420949e4011de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:01.322903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.326291 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.331794 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1372/2711] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:01 2022] DELETE /identity/v3/projects/b54e396f5d804768af420949e4011de9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:01.333436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:01.365199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a364ac-bcde-4ff2-adfe-6d2e3538200d service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:01.484429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:01.487185 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:01.491346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:01.492079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] PATH_INFO: `/v3/users/9b40a622d1a64d55ac355a2507b48698` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:01.494603 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b40a622d1a64d55ac355a2507b48698)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:01.497345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:01.607123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b40a622d1a64d55ac355a2507b48698'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.624881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:01.626426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:01.627264 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b40a622d1a64d55ac355a2507b48698'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.639521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b40a622d1a64d55ac355a2507b48698'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.640242 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b40a622d1a64d55ac355a2507b48698'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.640946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b40a622d1a64d55ac355a2507b48698'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.647065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b6dc-2cce-4e5b-8d52-925fcec8b461 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:01.653944 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:01.654725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:01.655079 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:01.655462 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] PATH_INFO: `/v3/users/cfc4077ebd9d4e05960fb68cc2406a87` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:01.657694 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc4077ebd9d4e05960fb68cc2406a87)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:01.660543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:01.664782 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1340/2712] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:01 2022] DELETE /identity/v3/users/9b40a622d1a64d55ac355a2507b48698 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:01.734195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc4077ebd9d4e05960fb68cc2406a87'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.734967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc4077ebd9d4e05960fb68cc2406a87'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.745318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc4077ebd9d4e05960fb68cc2406a87'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.746109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc4077ebd9d4e05960fb68cc2406a87'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.757055 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc4077ebd9d4e05960fb68cc2406a87'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:01.782232 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1373/2713] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:01 2022] DELETE /identity/v3/users/cfc4077ebd9d4e05960fb68cc2406a87 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:02.005190 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.007265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:02.084193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94566d5c-9088-4880-b321-de3785e7bf89 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.102401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:02.149499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.150141 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.150427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.150701 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] PATH_INFO: `/v3/projects/90f7baeae70a48caaa460359e9a3ac54` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.151828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f7baeae70a48caaa460359e9a3ac54)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.153797 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.177601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.180083 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:02.197509 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e21043-7d2d-4bf9-a8af-a83846f6c0a9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.225871 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f7baeae70a48caaa460359e9a3ac54', 'request_id': 'req-60181817-fe3a-4a5f-b321-888fb9a54b05'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.238573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:02.241695 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f7baeae70a48caaa460359e9a3ac54', 'request_id': 'req-60181817-fe3a-4a5f-b321-888fb9a54b05'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.256338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f7baeae70a48caaa460359e9a3ac54', 'request_id': 'req-60181817-fe3a-4a5f-b321-888fb9a54b05'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.277891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] The token cache is being invalidate because project 90f7baeae70a48caaa460359e9a3ac54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:02.278318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.279129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-60181817-fe3a-4a5f-b321-888fb9a54b05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.280894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1341/2714] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:02 2022] DELETE /identity/v3/projects/90f7baeae70a48caaa460359e9a3ac54 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:02.288036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a40e04-4e35-403c-b689-47aebf46ba7a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.288486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a40e04-4e35-403c-b689-47aebf46ba7a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.288850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a40e04-4e35-403c-b689-47aebf46ba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.312992 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-43a40e04-4e35-403c-b689-47aebf46ba7a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:02.314286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-43a40e04-4e35-403c-b689-47aebf46ba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:02.321774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.322433 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.322710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.322989 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] PATH_INFO: `/v3/projects/862629108fd34417bed67e1b5eedbe92` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.324197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=862629108fd34417bed67e1b5eedbe92)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.326527 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.384946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862629108fd34417bed67e1b5eedbe92', 'request_id': 'req-6129b021-1c45-49e9-b378-204f7bccacf8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.387455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1342/2715] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:02.391920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862629108fd34417bed67e1b5eedbe92', 'request_id': 'req-6129b021-1c45-49e9-b378-204f7bccacf8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.397484 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6cf2-ded9-4ed6-aa09-d11cdde9917f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.397843 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6cf2-ded9-4ed6-aa09-d11cdde9917f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.398099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f6cf2-ded9-4ed6-aa09-d11cdde9917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.398651 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862629108fd34417bed67e1b5eedbe92', 'request_id': 'req-6129b021-1c45-49e9-b378-204f7bccacf8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.414031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] The token cache is being invalidate because project 862629108fd34417bed67e1b5eedbe92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:02.414333 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.415109 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.419667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1374/2716] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:02 2022] DELETE /identity/v3/projects/862629108fd34417bed67e1b5eedbe92 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:02.420506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e49f6cf2-ded9-4ed6-aa09-d11cdde9917f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:02.420730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.422140 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:02.422371 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e49f6cf2-ded9-4ed6-aa09-d11cdde9917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:02.473984 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:02.492994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129b021-1c45-49e9-b378-204f7bccacf8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.494248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1343/2717] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:02.498799 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f6cf2-ded9-4ed6-aa09-d11cdde9917f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.499281 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.500006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.500358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.500702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] PATH_INFO: `/v3/users/80b53c816b034819a09e4acd18c598c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.502002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b53c816b034819a09e4acd18c598c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.505174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.509454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f6cf2-ded9-4ed6-aa09-d11cdde9917f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.515005 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.516165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.516454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.516735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.517473 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:02.517903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.519717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.537575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1344/2718] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:02.539130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.552307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f71ec-94ba-481f-b052-5b7d1dd13115 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.557964 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.558303 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.558655 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.559018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.559796 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.562284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.573851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b53c816b034819a09e4acd18c598c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.574666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b53c816b034819a09e4acd18c598c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.583897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:02.587808 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b53c816b034819a09e4acd18c598c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.588802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b53c816b034819a09e4acd18c598c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.600272 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b53c816b034819a09e4acd18c598c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:02.604262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1345/2719] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:02 2022] POST /identity/v3/projects => generated 361 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:02.605982 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.608849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:02.626486 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1375/2720] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:02 2022] DELETE /identity/v3/users/80b53c816b034819a09e4acd18c598c3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:02.702754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be82a76-3d55-4d11-a444-dd063959f2d6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.767720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.768367 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.769661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.769985 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.770726 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.773364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.805749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1346/2721] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:02 2022] POST /identity/v3/users => generated 281 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:02.811247 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.837854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e10f-1ab3-4ff4-88d9-74ca0177d4f0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.845979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.852146 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.852548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.852925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.853700 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.855467 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.867246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1376/2722] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:02.872099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.884806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec291e-7180-498a-b3a5-16449d016f55 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.890225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.890609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.890892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.891165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.891960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.894881 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.911040 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1347/2723] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:02.912732 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.936222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a98333-f3fd-4ca2-b321-2ef65afea92e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.942897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a708af7-24fb-4f41-a3bc-b2f90055d5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.943767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a708af7-24fb-4f41-a3bc-b2f90055d5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.944153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a708af7-24fb-4f41-a3bc-b2f90055d5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.944593 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a708af7-24fb-4f41-a3bc-b2f90055d5e4 admin admin] PATH_INFO: `/v3/projects/6357e3bc4f9940caa1e9bb87c9136051/users/de906be9155b4bc999f5a093938bf834/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.957621 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:02.960953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:02.971952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a708af7-24fb-4f41-a3bc-b2f90055d5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6357e3bc4f9940caa1e9bb87c9136051, user_id=de906be9155b4bc999f5a093938bf834, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.976404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a708af7-24fb-4f41-a3bc-b2f90055d5e4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.977631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e13115-aeba-4909-9c03-976df0833ff7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:02.984194 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:02.984963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:02.985513 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:02.985897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] PATH_INFO: `/v3/projects/59f74258d2804930a74fb18db98f4d92` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:02.987351 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=59f74258d2804930a74fb18db98f4d92)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:02.989569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:02.993012 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1377/2724] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:02 2022] PUT /identity/v3/projects/6357e3bc4f9940caa1e9bb87c9136051/users/de906be9155b4bc999f5a093938bf834/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:03.004893 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.005187 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.005493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.038332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] MFA Rules not processed for user `de906be9155b4bc999f5a093938bf834`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:03.039549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.043216 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:03.057590 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59f74258d2804930a74fb18db98f4d92', 'request_id': 'req-83a5a000-9252-4a89-be0e-74c62eed5a2b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.064101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59f74258d2804930a74fb18db98f4d92', 'request_id': 'req-83a5a000-9252-4a89-be0e-74c62eed5a2b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.071154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59f74258d2804930a74fb18db98f4d92', 'request_id': 'req-83a5a000-9252-4a89-be0e-74c62eed5a2b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.086764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] The token cache is being invalidate because project 59f74258d2804930a74fb18db98f4d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:03.087026 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.087573 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-83a5a000-9252-4a89-be0e-74c62eed5a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.089121 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1348/2725] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:02 2022] DELETE /identity/v3/projects/59f74258d2804930a74fb18db98f4d92 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:03.094019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649aca3-7976-4b41-bc90-d93ba3fc0575 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.094307 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649aca3-7976-4b41-bc90-d93ba3fc0575 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.094584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649aca3-7976-4b41-bc90-d93ba3fc0575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.120599 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b649aca3-7976-4b41-bc90-d93ba3fc0575 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:03.121971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b649aca3-7976-4b41-bc90-d93ba3fc0575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.131002 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1378/2726] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:03.174044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.176664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.222627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1349/2727] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:03.224936 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47e478-57f4-4390-a231-6011ba184dfb None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.226257 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47e478-57f4-4390-a231-6011ba184dfb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.227080 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47e478-57f4-4390-a231-6011ba184dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.248626 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7e47e478-57f4-4390-a231-6011ba184dfb None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:03.249865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7e47e478-57f4-4390-a231-6011ba184dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.253482 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0ad1b-f628-45b5-9f45-5c9db190920e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.257141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.257761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.258011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1350/2728] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:03.258141 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.258304 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.262055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47e478-57f4-4390-a231-6011ba184dfb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.267166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.269082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.271086 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47e478-57f4-4390-a231-6011ba184dfb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.275831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.276426 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.276743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.277023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.277660 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:03.277973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.278620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1379/2729] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:03.279705 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.292250 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1351/2730] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:51:03.294156 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.295674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.296959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.307259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9318123-dffc-47e6-851e-3c7c064c4363 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.309715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbe3fe-0b46-4fb9-a507-c08015a30458 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.312169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.312764 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.313044 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.313360 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.314016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.315008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.315833 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.316035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.316260 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.316525 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] PATH_INFO: `/v3/users/de906be9155b4bc999f5a093938bf834` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.317675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de906be9155b4bc999f5a093938bf834)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.319666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.343248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1352/2731] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:03 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:03.343523 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:03.348778 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.374961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de906be9155b4bc999f5a093938bf834'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.376060 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de906be9155b4bc999f5a093938bf834'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.383637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de906be9155b4bc999f5a093938bf834'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.384358 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de906be9155b4bc999f5a093938bf834'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.392047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca6d5db-b85b-41d9-b461-ce7e1a42cb05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de906be9155b4bc999f5a093938bf834'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:03.410791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1380/2732] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:03 2022] DELETE /identity/v3/users/de906be9155b4bc999f5a093938bf834 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:03.415095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.415547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.415831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.415988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b9545-38dc-43c2-82b8-7a4026adc190 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.435505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:03.436730 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.445409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:03.476339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.477236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.477517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.477791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.478432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.480439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.499289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1381/2733] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:03.507716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1353/2734] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:03 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:03.509016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.509246 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.510465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.511014 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.522962 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27971ef-c5a1-440a-83d3-a38fb310816e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.527655 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.528267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.528548 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.528825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.529483 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.531093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.543510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1382/2735] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:03.547445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.558249 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ebe5f-cc6b-46d1-8025-e592761bc139 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.563921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.564583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.564854 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.565135 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.565817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.567495 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.573668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fc6dc-df54-4ddd-af6d-d297e14cee41 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.577359 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febf13d-7b7f-4283-9ac5-f924a48f59e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.577537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1383/2736] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:03.578222 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febf13d-7b7f-4283-9ac5-f924a48f59e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.578575 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febf13d-7b7f-4283-9ac5-f924a48f59e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.578915 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febf13d-7b7f-4283-9ac5-f924a48f59e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.581684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.588918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febf13d-7b7f-4283-9ac5-f924a48f59e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.590432 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febf13d-7b7f-4283-9ac5-f924a48f59e2 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.591152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988371e-ea4c-4949-926b-f58eb685efb4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.596190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.596921 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.597209 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.597563 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] PATH_INFO: `/v3/projects/5386662ca607431ba8e0ccf55c22f962/users/a5410737c6fd4875a8377796ee594c8d/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.601475 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1354/2737] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:03.613584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5386662ca607431ba8e0ccf55c22f962, user_id=a5410737c6fd4875a8377796ee594c8d, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.616455 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.633118 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1384/2738] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:03 2022] PUT /identity/v3/projects/5386662ca607431ba8e0ccf55c22f962/users/a5410737c6fd4875a8377796ee594c8d/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:03.643598 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f573f5-d7c8-413c-ad9f-3bcfdcf59c49 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.644099 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f573f5-d7c8-413c-ad9f-3bcfdcf59c49 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.644591 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f573f5-d7c8-413c-ad9f-3bcfdcf59c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.677864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d5f573f5-d7c8-413c-ad9f-3bcfdcf59c49 None None] MFA Rules not processed for user `a5410737c6fd4875a8377796ee594c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:03.679133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d5f573f5-d7c8-413c-ad9f-3bcfdcf59c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.744559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1355/2739] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:03.757204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.759404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.820244 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d516462-642f-42e4-a237-c20ab7d42587 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.825899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.826726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.827018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.827332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.834068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:03.836765 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.839265 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.850344 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1385/2740] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:03.870506 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f573f5-d7c8-413c-ad9f-3bcfdcf59c49 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.896982 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.898449 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:03.931631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:03.945453 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f573f5-d7c8-413c-ad9f-3bcfdcf59c49 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.956204 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.956858 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.957133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.957407 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] PATH_INFO: `/v3/users/a5410737c6fd4875a8377796ee594c8d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.958840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5410737c6fd4875a8377796ee594c8d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.961136 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.985823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccd160-b9b0-432e-a743-69b759c4116e service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.993296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.994074 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.994343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.994546 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] PATH_INFO: `/v3/projects/6357e3bc4f9940caa1e9bb87c9136051` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.996321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6357e3bc4f9940caa1e9bb87c9136051)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.998474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.026600 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5410737c6fd4875a8377796ee594c8d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.034169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5410737c6fd4875a8377796ee594c8d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.042722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5410737c6fd4875a8377796ee594c8d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.043562 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5410737c6fd4875a8377796ee594c8d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.044622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0752a741-58a5-4cd3-a4fa-1158a0717c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5410737c6fd4875a8377796ee594c8d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.057903 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6357e3bc4f9940caa1e9bb87c9136051', 'request_id': 'req-9b5f9922-0c94-42a8-8f49-eb972a710705'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.065359 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6357e3bc4f9940caa1e9bb87c9136051', 'request_id': 'req-9b5f9922-0c94-42a8-8f49-eb972a710705'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.067632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1356/2741] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:03 2022] DELETE /identity/v3/users/a5410737c6fd4875a8377796ee594c8d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:04.072110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6357e3bc4f9940caa1e9bb87c9136051', 'request_id': 'req-9b5f9922-0c94-42a8-8f49-eb972a710705'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.072722 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.073055 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.073287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.087296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] The token cache is being invalidate because project 6357e3bc4f9940caa1e9bb87c9136051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:04.087598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.088243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9b5f9922-0c94-42a8-8f49-eb972a710705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.090222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1386/2742] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:03 2022] DELETE /identity/v3/projects/6357e3bc4f9940caa1e9bb87c9136051 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:04.096046 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a1b5c-1249-44bc-8c7c-851e2e491e68 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.097088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a1b5c-1249-44bc-8c7c-851e2e491e68 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.097343 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a1b5c-1249-44bc-8c7c-851e2e491e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.101457 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:04.102825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.123423 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-194a1b5c-1249-44bc-8c7c-851e2e491e68 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:04.124823 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-194a1b5c-1249-44bc-8c7c-851e2e491e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.170734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1357/2743] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:04.181213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.182678 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.196112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1387/2744] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:04.205189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a788e8d-79bb-41a6-8523-cde08055ee33 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.205642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a788e8d-79bb-41a6-8523-cde08055ee33 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.205933 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a788e8d-79bb-41a6-8523-cde08055ee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.226403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2a788e8d-79bb-41a6-8523-cde08055ee33 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:04.227683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2a788e8d-79bb-41a6-8523-cde08055ee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.236085 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1388/2745] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:04.240559 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a788e8d-79bb-41a6-8523-cde08055ee33 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.242496 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069eac3-fa73-44b6-b12d-a76196e8cb89 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.247642 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.248339 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.248680 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.249047 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.250571 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a788e8d-79bb-41a6-8523-cde08055ee33 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.256334 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.256970 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.257254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.257533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.258329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:04.258639 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.259404 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.260317 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.260959 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.270619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1358/2746] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:04.273023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1389/2747] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:51:04.277277 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.287828 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a4a90-1f35-43b5-8323-357cc8a7072c service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.292167 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.292824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.293108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.293384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.294025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.295981 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.312441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:04.328550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1359/2748] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:04.333117 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.344171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a49a18-9dcd-4378-aaf7-8a6d7095bd29 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.348745 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:04.406852 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.407607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.408008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.408413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.409537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.411514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.434915 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1390/2749] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:04.439624 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.450434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e387-06e6-43de-b1ce-f3d06dc44946 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.455954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.456576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.456859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.457133 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.457790 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.459603 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.470083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1360/2750] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:04.474973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.485597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6d79f-eb22-463b-af4f-ce57cc6e6421 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.491144 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.491822 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.492103 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.492375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.493036 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.494659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.507542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1391/2751] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:04.511879 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.523364 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0b8ca-e73b-4e11-bd48-ac9138095295 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.529046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961c87b-2948-45d5-b8df-42c2ce7859c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.529808 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961c87b-2948-45d5-b8df-42c2ce7859c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.530062 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961c87b-2948-45d5-b8df-42c2ce7859c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.530378 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961c87b-2948-45d5-b8df-42c2ce7859c5 admin admin] PATH_INFO: `/v3/projects/e9e96eecdb3d4498b0cc18846ebd5794/users/cf0c764c2595424fba631d573f5b6883/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.548895 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961c87b-2948-45d5-b8df-42c2ce7859c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e96eecdb3d4498b0cc18846ebd5794, user_id=cf0c764c2595424fba631d573f5b6883, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.551703 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961c87b-2948-45d5-b8df-42c2ce7859c5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.554902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.556911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.572130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1361/2752] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:04 2022] PUT /identity/v3/projects/e9e96eecdb3d4498b0cc18846ebd5794/users/cf0c764c2595424fba631d573f5b6883/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:04.574190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e787bbb-71b7-40eb-8416-e3ee3a4028e5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.584401 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.584830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.585298 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.614059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] MFA Rules not processed for user `cf0c764c2595424fba631d573f5b6883`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:04.615435 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.641580 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.642228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.642641 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.642929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] PATH_INFO: `/v3/projects/5386662ca607431ba8e0ccf55c22f962` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.644250 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5386662ca607431ba8e0ccf55c22f962)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.646226 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.681923 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1362/2753] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:04.694179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5386662ca607431ba8e0ccf55c22f962', 'request_id': 'req-7ff6d969-2b7d-4d03-939f-321a00d376db'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.695891 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.697322 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.700610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5386662ca607431ba8e0ccf55c22f962', 'request_id': 'req-7ff6d969-2b7d-4d03-939f-321a00d376db'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.706704 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5386662ca607431ba8e0ccf55c22f962', 'request_id': 'req-7ff6d969-2b7d-4d03-939f-321a00d376db'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.720789 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] The token cache is being invalidate because project 5386662ca607431ba8e0ccf55c22f962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:04.721321 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.721988 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7ff6d969-2b7d-4d03-939f-321a00d376db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:04.726905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1392/2754] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:04 2022] DELETE /identity/v3/projects/5386662ca607431ba8e0ccf55c22f962 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:04.728376 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f3aa9-1ad0-4fc4-b241-0c0e35ec43ca None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.728736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f3aa9-1ad0-4fc4-b241-0c0e35ec43ca None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.729088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f3aa9-1ad0-4fc4-b241-0c0e35ec43ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.755770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-de7f3aa9-1ad0-4fc4-b241-0c0e35ec43ca None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:04.758030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-de7f3aa9-1ad0-4fc4-b241-0c0e35ec43ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.761738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac68a3e-4fc2-49f4-a76b-b51bccbf27be None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.769767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:04.815848 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380786eb9b7848d59f3bb0007726dd2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.816462 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.816741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.817011 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.818824 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.824575 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1393/2755] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:04.832837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff512b-c637-410b-91e5-7e81323eba29 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.833054 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff512b-c637-410b-91e5-7e81323eba29 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.833332 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff512b-c637-410b-91e5-7e81323eba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.836908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:04.853996 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5eff512b-c637-410b-91e5-7e81323eba29 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:04.856164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5eff512b-c637-410b-91e5-7e81323eba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.868279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1394/2756] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:04.872508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff512b-c637-410b-91e5-7e81323eba29 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.879286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.880733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.883280 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff512b-c637-410b-91e5-7e81323eba29 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.888076 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.888672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.888952 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.889223 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.889731 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1363/2757] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:04.889873 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:04.890196 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.891849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.904897 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1395/2758] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:51:04.909810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.920921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cb487-8289-4a6d-8f1b-c6826c3d4afd service glance] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.926508 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.927114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.927430 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.927706 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.928341 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.930243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.960888 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1364/2759] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:04.965184 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.976308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d6178-f9ba-453f-aada-39913c296cbc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:05.046752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:05.047411 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:05.047699 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:05.047979 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:05.048623 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:05.050591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:05.073656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1396/2760] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:04 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:05.077921 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:05.088581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3c2ca-1172-46f8-8b84-320682fbe19f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:05.093437 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:05.094048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:05.094333 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:05.094610 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:05.095345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:05.097023 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:05.108188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1365/2761] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:05.112384 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:05.122858 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087e0e-3e8c-4b11-8a81-568dd4c5c53b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:05.128341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e930c20-eda3-4aea-95bb-6050c59adb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:05.128949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e930c20-eda3-4aea-95bb-6050c59adb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:05.129228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e930c20-eda3-4aea-95bb-6050c59adb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:05.129502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e930c20-eda3-4aea-95bb-6050c59adb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:05.130152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e930c20-eda3-4aea-95bb-6050c59adb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:05.131800 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e930c20-eda3-4aea-95bb-6050c59adb9a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:05.142851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1397/2762] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:05.147431 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:05.157667 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348218a5-a28d-43d6-8e6f-16cfd6a8206a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:05.164255 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:05.164925 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:05.165238 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:05.165481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] PATH_INFO: `/v3/projects/4f0ae15cde5c4f47a1a175169708a764/users/6413a3a8bced450b9fc641af23cf2bca/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:05.182578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0ae15cde5c4f47a1a175169708a764, user_id=6413a3a8bced450b9fc641af23cf2bca, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:05.185780 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:05.200231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1366/2763] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:05 2022] PUT /identity/v3/projects/4f0ae15cde5c4f47a1a175169708a764/users/6413a3a8bced450b9fc641af23cf2bca/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:05.225957 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:05.226268 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:05.226547 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:05.261364 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] MFA Rules not processed for user `6413a3a8bced450b9fc641af23cf2bca`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:05.262918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:05.373121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1398/2764] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:05.384718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:05.386414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:05.452311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67134f-815a-441d-a2ff-11fc4f7d8bdf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:05.456233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380786eb9b7848d59f3bb0007726dd2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:05.456830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:05.457108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:05.457380 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:05.465493 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:05.466932 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:05.475612 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1367/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:06.079476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:06.081179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:06.088229 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:06.145354 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fe6ae-360b-4a07-bd18-59df5e9bffb2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:06.149016 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:06.149626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:06.149911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:06.150185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:06.151963 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:06.228216 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:06.230098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:06.242072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1399/2766] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:15.471804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=69,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:16.237481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=48 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:17.555245 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:17.557199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:17.654156 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7936fb-8939-4511-a7fc-17ef6d6c704a service glance] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:17.658231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c901f-ca63-45f9-b3a4-7e4377821b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:17.658877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c901f-ca63-45f9-b3a4-7e4377821b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:17.659179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c901f-ca63-45f9-b3a4-7e4377821b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:17.659478 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c901f-ca63-45f9-b3a4-7e4377821b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:17.661240 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5b3c901f-ca63-45f9-b3a4-7e4377821b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:17.755765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c901f-ca63-45f9-b3a4-7e4377821b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:17.757346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c901f-ca63-45f9-b3a4-7e4377821b22 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:17.769363 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1368/2767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:27.763546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=72 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:31.508751 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:31.510980 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:31.658204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bcb9d-293a-48d0-8978-18a6c9f62556 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:31.663570 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:31.664221 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:31.664496 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:31.664764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] PATH_INFO: `/v3/users/3e9b2d5386a444ed93c35708cd7b5666` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:31.665932 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e9b2d5386a444ed93c35708cd7b5666)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:31.667934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:31.718422 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e9b2d5386a444ed93c35708cd7b5666'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:31.719131 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e9b2d5386a444ed93c35708cd7b5666'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:31.726230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e9b2d5386a444ed93c35708cd7b5666'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:31.726850 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e9b2d5386a444ed93c35708cd7b5666'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:31.736194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e9b2d5386a444ed93c35708cd7b5666'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:31.756238 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1400/2768] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:31 2022] DELETE /identity/v3/users/3e9b2d5386a444ed93c35708cd7b5666 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:31.760965 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:31.761455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:31.761572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:31.790101 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:31.791519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:31.852976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1369/2769] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:31.863261 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:31.865128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:31.891793 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:31.934576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de17a9-7d17-4a5b-b9b6-835df6f58009 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:31.938713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:31.939371 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:31.939659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:31.939936 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:31.950256 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:31.951792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:31.961972 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1401/2770] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:32.244464 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:32.245945 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:32.259569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e5fe-f3b2-4a2f-bdcb-85932a18baa5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:32.265221 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:32.265877 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:32.266158 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:32.266447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] PATH_INFO: `/v3/projects/eaae9976808d4eb994cddc6251948c2e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:32.267741 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaae9976808d4eb994cddc6251948c2e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:32.269737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:32.320031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaae9976808d4eb994cddc6251948c2e', 'request_id': 'req-fc75ecbf-efc1-44b9-944d-93af3708b3e1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.325656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaae9976808d4eb994cddc6251948c2e', 'request_id': 'req-fc75ecbf-efc1-44b9-944d-93af3708b3e1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.332036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaae9976808d4eb994cddc6251948c2e', 'request_id': 'req-fc75ecbf-efc1-44b9-944d-93af3708b3e1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.349025 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] The token cache is being invalidate because project eaae9976808d4eb994cddc6251948c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:32.349409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.350131 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.351714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1370/2771] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:32 2022] DELETE /identity/v3/projects/eaae9976808d4eb994cddc6251948c2e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:32.355981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:32.369476 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f58a8b-93bd-4484-8e5a-1b2223c0b1f7 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:32.442023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:32.442908 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:32.443444 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:32.443591 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] PATH_INFO: `/v3/users/b6c6d0bcb55740d59e5e61713d3ccfdf` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:32.444712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c6d0bcb55740d59e5e61713d3ccfdf)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:32.446674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:32.494827 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c6d0bcb55740d59e5e61713d3ccfdf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.495511 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c6d0bcb55740d59e5e61713d3ccfdf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.502189 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c6d0bcb55740d59e5e61713d3ccfdf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.502801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c6d0bcb55740d59e5e61713d3ccfdf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.511370 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e90b54a7-c930-41db-840c-48e5089adc3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c6d0bcb55740d59e5e61713d3ccfdf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.528861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1402/2772] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:32 2022] DELETE /identity/v3/users/b6c6d0bcb55740d59e5e61713d3ccfdf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:32.821755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:32.823504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:32.840546 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75ecbf-efc1-44b9-944d-93af3708b3e1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:32.903989 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:32.904639 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:32.904929 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:32.905198 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] PATH_INFO: `/v3/projects/3d069f5657a44b1eaa17f2cebc6c835b` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:32.906348 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d069f5657a44b1eaa17f2cebc6c835b)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:32.908300 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:32.933065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:32.964270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d069f5657a44b1eaa17f2cebc6c835b', 'request_id': 'req-b834666f-aa31-4657-ad04-9c84e82f3644'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.971842 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d069f5657a44b1eaa17f2cebc6c835b', 'request_id': 'req-b834666f-aa31-4657-ad04-9c84e82f3644'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.980543 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d069f5657a44b1eaa17f2cebc6c835b', 'request_id': 'req-b834666f-aa31-4657-ad04-9c84e82f3644'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.997749 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] The token cache is being invalidate because project 3d069f5657a44b1eaa17f2cebc6c835b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:32.998046 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:32.998755 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b834666f-aa31-4657-ad04-9c84e82f3644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:33.000376 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1371/2773] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:32 2022] DELETE /identity/v3/projects/3d069f5657a44b1eaa17f2cebc6c835b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:33.006357 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68f23a-5502-43c4-a5bd-6eee91c4eeca None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.006642 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68f23a-5502-43c4-a5bd-6eee91c4eeca None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.006919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68f23a-5502-43c4-a5bd-6eee91c4eeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.058510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ed68f23a-5502-43c4-a5bd-6eee91c4eeca None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:33.073078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ed68f23a-5502-43c4-a5bd-6eee91c4eeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:33.076048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:33.144286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1403/2774] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:33.153187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35e30b-0337-468c-88f8-4bad00a0fef1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.153511 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35e30b-0337-468c-88f8-4bad00a0fef1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.153789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35e30b-0337-468c-88f8-4bad00a0fef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.176003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dd35e30b-0337-468c-88f8-4bad00a0fef1 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:33.177285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dd35e30b-0337-468c-88f8-4bad00a0fef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:33.186911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1372/2775] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:33.191362 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68f23a-5502-43c4-a5bd-6eee91c4eeca None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:33.202722 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68f23a-5502-43c4-a5bd-6eee91c4eeca None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:33.208712 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:33.209308 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.209598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.210237 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.211583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:33.211909 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:33.213561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:33.235841 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1404/2776] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:33.237270 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35e30b-0337-468c-88f8-4bad00a0fef1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:33.247429 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35e30b-0337-468c-88f8-4bad00a0fef1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:33.252545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:33.253165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.253454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.253743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.254466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:33.256408 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:33.291517 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1373/2777] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:33 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:33.296705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:33.307531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e987137-e2c5-456b-8ddd-4b9cfe4056be admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:33.365386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:33.366027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.366353 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.366612 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.367251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:33.369185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:33.391659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1405/2778] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:33 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:33.396092 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:33.407788 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb532e-fabc-4d47-b4dc-e747f886543d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:33.412993 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:33.413832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.414114 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.414387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.415032 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:33.416661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:33.426643 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1374/2779] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:33.431100 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:33.441597 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ab2fe-e44e-4e0c-9ab9-48886deeae86 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:33.446880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f6159-dee0-4d16-96f7-2146e002fda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:33.447512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f6159-dee0-4d16-96f7-2146e002fda9 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.447798 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f6159-dee0-4d16-96f7-2146e002fda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.448065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f6159-dee0-4d16-96f7-2146e002fda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.448710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f6159-dee0-4d16-96f7-2146e002fda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:33.450301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60f6159-dee0-4d16-96f7-2146e002fda9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:33.461007 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1406/2780] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:33.465369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:33.475995 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcee5c-8478-4301-9b6d-07534a34fbe6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:33.480919 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:33.481583 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.481865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.482146 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] PATH_INFO: `/v3/projects/8f3a72628fd04c489680ab5cb90b4054/users/c5923f09ffda49c29cd0af9c226edf2a/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.498771 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3a72628fd04c489680ab5cb90b4054, user_id=c5923f09ffda49c29cd0af9c226edf2a, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:33.501582 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:33.521698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1375/2781] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:33 2022] PUT /identity/v3/projects/8f3a72628fd04c489680ab5cb90b4054/users/c5923f09ffda49c29cd0af9c226edf2a/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:33.531839 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.532331 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.532564 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.561925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] MFA Rules not processed for user `c5923f09ffda49c29cd0af9c226edf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:33.563233 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:33.631242 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1407/2782] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:33.642262 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:33.644070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:33.703112 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4c028-dfab-4a16-92da-3372c380c512 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:33.707004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bac54-cdad-43a8-bcc1-fd6adf0269b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:33.707683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bac54-cdad-43a8-bcc1-fd6adf0269b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:33.707973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bac54-cdad-43a8-bcc1-fd6adf0269b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:33.708249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bac54-cdad-43a8-bcc1-fd6adf0269b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:33.719127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bac54-cdad-43a8-bcc1-fd6adf0269b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:33.721186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bac54-cdad-43a8-bcc1-fd6adf0269b3 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:33.730961 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1376/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:34.903259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:34.905573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:34.922611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:34.979113 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892079-7776-4ddb-84f2-eaa138c818af None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:34.984288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:34.984918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:34.985228 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:34.985500 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] PATH_INFO: `/v3/users/6413a3a8bced450b9fc641af23cf2bca` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:34.986653 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6413a3a8bced450b9fc641af23cf2bca)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:34.988714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:35.037967 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6413a3a8bced450b9fc641af23cf2bca'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.038664 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6413a3a8bced450b9fc641af23cf2bca'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.045395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6413a3a8bced450b9fc641af23cf2bca'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.046010 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6413a3a8bced450b9fc641af23cf2bca'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.054589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6413a3a8bced450b9fc641af23cf2bca'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.073403 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1408/2784] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:34 2022] DELETE /identity/v3/users/6413a3a8bced450b9fc641af23cf2bca => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:35.077935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.078352 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.078455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.098306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:35.099504 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.144024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:35.164382 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1377/2785] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:35.175302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:35.176994 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.266953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3db92d-5814-4963-9180-d95564c30637 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:35.271594 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e18a3a-5580-4489-ba00-d7f4d745d5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:35.272270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e18a3a-5580-4489-ba00-d7f4d745d5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.272610 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e18a3a-5580-4489-ba00-d7f4d745d5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.272891 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e18a3a-5580-4489-ba00-d7f4d745d5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.284522 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e18a3a-5580-4489-ba00-d7f4d745d5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:35.286164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e18a3a-5580-4489-ba00-d7f4d745d5a7 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:35.299682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1409/2786] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:35.588465 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:35.590791 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.608039 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a31e8-e768-43a5-9337-b4e9bdee4375 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:35.612954 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:35.613668 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.614013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.614990 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] PATH_INFO: `/v3/projects/4f0ae15cde5c4f47a1a175169708a764` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.617336 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0ae15cde5c4f47a1a175169708a764)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:35.619652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:35.670243 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0ae15cde5c4f47a1a175169708a764', 'request_id': 'req-f9fa140b-9b95-49dd-871d-9a85edae1cc3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.675802 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0ae15cde5c4f47a1a175169708a764', 'request_id': 'req-f9fa140b-9b95-49dd-871d-9a85edae1cc3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.681976 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0ae15cde5c4f47a1a175169708a764', 'request_id': 'req-f9fa140b-9b95-49dd-871d-9a85edae1cc3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.695630 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] The token cache is being invalidate because project 4f0ae15cde5c4f47a1a175169708a764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:35.695918 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.696558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f9fa140b-9b95-49dd-871d-9a85edae1cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:35.698067 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1378/2787] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:35 2022] DELETE /identity/v3/projects/4f0ae15cde5c4f47a1a175169708a764 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:35.703145 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103e6a-43fc-4c1f-9137-a7714bf73883 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.703464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103e6a-43fc-4c1f-9137-a7714bf73883 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.703740 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103e6a-43fc-4c1f-9137-a7714bf73883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.724056 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-96103e6a-43fc-4c1f-9137-a7714bf73883 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:35.725394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-96103e6a-43fc-4c1f-9137-a7714bf73883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.734902 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:35.787875 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1410/2788] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:35.796193 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf77bf-19ed-48c8-97df-b54bab5fe85f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.796514 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf77bf-19ed-48c8-97df-b54bab5fe85f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.796789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf77bf-19ed-48c8-97df-b54bab5fe85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.817272 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-acdf77bf-19ed-48c8-97df-b54bab5fe85f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:35.818469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-acdf77bf-19ed-48c8-97df-b54bab5fe85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:35.827134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1379/2789] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:35.831485 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103e6a-43fc-4c1f-9137-a7714bf73883 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:35.842220 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103e6a-43fc-4c1f-9137-a7714bf73883 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:35.848447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:35.849077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.849361 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.849635 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.850456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:35.850917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:35.852288 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:35.864458 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1411/2790] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:35.869541 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf77bf-19ed-48c8-97df-b54bab5fe85f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:35.878835 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf77bf-19ed-48c8-97df-b54bab5fe85f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:35.884085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:35.884677 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:35.884955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:35.885227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:35.885841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:35.887763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:35.918164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1380/2791] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:35 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:35.922831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:35.940210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55868d21-f882-405a-bd20-e8c7dab604b7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:36.003493 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:36.004101 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.004380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.004676 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.005301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:36.007246 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:36.034626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1412/2792] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:35 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:36.040413 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.052123 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aaa5b-c5a3-4f23-bf5f-30a127c3acbb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:36.058389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:36.059261 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.059631 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.060390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.061183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:36.063860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:36.080227 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1381/2793] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:36.085986 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.097497 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13261bd-117f-49e5-89fe-4b0a705cc5d8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:36.103715 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce091f-f6ee-49a8-995d-9d1dd0c4ccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:36.104315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce091f-f6ee-49a8-995d-9d1dd0c4ccde admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.104605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce091f-f6ee-49a8-995d-9d1dd0c4ccde admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.104881 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce091f-f6ee-49a8-995d-9d1dd0c4ccde admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.105532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce091f-f6ee-49a8-995d-9d1dd0c4ccde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:36.107158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce091f-f6ee-49a8-995d-9d1dd0c4ccde admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:36.118404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1413/2794] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:36.122659 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.133271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad7c55-be34-4b0a-b92c-6d0908389a0f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:36.139070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:36.139759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.140043 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.140323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] PATH_INFO: `/v3/projects/8865b82e64624888bb3267ec67614a45/users/38bd2be0b9384ee4814175d908d24dd1/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.155739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8865b82e64624888bb3267ec67614a45, user_id=38bd2be0b9384ee4814175d908d24dd1, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:36.158414 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:36.173178 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1382/2795] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:36 2022] PUT /identity/v3/projects/8865b82e64624888bb3267ec67614a45/users/38bd2be0b9384ee4814175d908d24dd1/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:36.187857 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33de643-4e31-4882-87b6-52dae1176753 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.188119 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33de643-4e31-4882-87b6-52dae1176753 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.188425 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33de643-4e31-4882-87b6-52dae1176753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.219965 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c33de643-4e31-4882-87b6-52dae1176753 None None] MFA Rules not processed for user `38bd2be0b9384ee4814175d908d24dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:36.221207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c33de643-4e31-4882-87b6-52dae1176753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:36.295501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1414/2796] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:36.307460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.310997 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:36.386892 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c728ae4-9a09-43be-a093-af64cb787e79 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:36.391569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1d381-e980-4cb7-98a5-63c40fb44d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:36.392186 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d381-e980-4cb7-98a5-63c40fb44d30 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.392468 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d381-e980-4cb7-98a5-63c40fb44d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.392740 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d381-e980-4cb7-98a5-63c40fb44d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.402477 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1d381-e980-4cb7-98a5-63c40fb44d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:36.403944 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1d381-e980-4cb7-98a5-63c40fb44d30 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:36.414058 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1383/2797] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:36.440440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33de643-4e31-4882-87b6-52dae1176753 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.452374 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:36.510005 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33de643-4e31-4882-87b6-52dae1176753 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:36.515791 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:36.516427 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.516707 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.516977 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] PATH_INFO: `/v3/users/38bd2be0b9384ee4814175d908d24dd1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.518230 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38bd2be0b9384ee4814175d908d24dd1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:36.520231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:36.571019 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38bd2be0b9384ee4814175d908d24dd1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:36.571777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38bd2be0b9384ee4814175d908d24dd1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:36.578841 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38bd2be0b9384ee4814175d908d24dd1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:36.579524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38bd2be0b9384ee4814175d908d24dd1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:36.587660 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38bd2be0b9384ee4814175d908d24dd1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:36.606430 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1415/2798] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:36 2022] DELETE /identity/v3/users/38bd2be0b9384ee4814175d908d24dd1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:36.611164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.611527 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.611812 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.631897 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:36.633873 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:36.638859 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:36.703784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1384/2799] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:36.714173 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:36.715761 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:36.792428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321efbe0-7c12-4143-94e8-fef5dabf0d06 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:36.796217 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:36.796837 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:36.797150 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:36.797404 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:36.806879 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:36.808441 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:36.819030 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1416/2800] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:37.097970 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:37.099474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:37.114750 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43645e2-b080-4422-9981-98a8d0ddb6fa None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:37.125662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:37.125662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:37.125662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:37.125662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] PATH_INFO: `/v3/projects/8865b82e64624888bb3267ec67614a45` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:37.126533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8865b82e64624888bb3267ec67614a45)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:37.127727 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:37.196867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8865b82e64624888bb3267ec67614a45', 'request_id': 'req-33253385-72fe-4bb8-b01f-fd21e594a3d5'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:37.202793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8865b82e64624888bb3267ec67614a45', 'request_id': 'req-33253385-72fe-4bb8-b01f-fd21e594a3d5'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:37.209287 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8865b82e64624888bb3267ec67614a45', 'request_id': 'req-33253385-72fe-4bb8-b01f-fd21e594a3d5'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:37.226688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] The token cache is being invalidate because project 8865b82e64624888bb3267ec67614a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:37.227012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:37.227708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33253385-72fe-4bb8-b01f-fd21e594a3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:37.229327 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1385/2801] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:37 2022] DELETE /identity/v3/projects/8865b82e64624888bb3267ec67614a45 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:43.297726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:43.299777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:43.314650 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60a1a1-5d90-4752-903d-51551ca3004f service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:43.477492 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:43.509691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:43.510417 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.510758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.511121 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] PATH_INFO: `/v3/users/c5923f09ffda49c29cd0af9c226edf2a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.512393 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5923f09ffda49c29cd0af9c226edf2a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:43.514325 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:43.563400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5923f09ffda49c29cd0af9c226edf2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:43.564026 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5923f09ffda49c29cd0af9c226edf2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:43.571267 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5923f09ffda49c29cd0af9c226edf2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:43.572027 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5923f09ffda49c29cd0af9c226edf2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:43.580607 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5923f09ffda49c29cd0af9c226edf2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:43.599065 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1417/2802] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:43 2022] DELETE /identity/v3/users/c5923f09ffda49c29cd0af9c226edf2a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:43.604162 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.604693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.604806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.624908 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:43.626118 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:43.697622 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1386/2803] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:43.708001 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:43.709536 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:43.769992 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2325d-69b9-48e9-8e8e-670c96920e1c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:43.774138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4ca8b-73b4-4a97-8215-c615f3703539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:43.774753 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4ca8b-73b4-4a97-8215-c615f3703539 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:43.775301 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4ca8b-73b4-4a97-8215-c615f3703539 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:43.775672 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4ca8b-73b4-4a97-8215-c615f3703539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:43.785050 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4ca8b-73b4-4a97-8215-c615f3703539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:43.786505 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4ca8b-73b4-4a97-8215-c615f3703539 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:43.796197 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1418/2804] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:44.083556 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.085035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:44.098454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65043cac-1836-4b46-8ad2-44bfae16d4d8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.104540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.105241 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.105550 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.105816 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] PATH_INFO: `/v3/projects/8f3a72628fd04c489680ab5cb90b4054` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.106986 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f3a72628fd04c489680ab5cb90b4054)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.108958 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.144860 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:44.161229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f3a72628fd04c489680ab5cb90b4054', 'request_id': 'req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:44.166840 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f3a72628fd04c489680ab5cb90b4054', 'request_id': 'req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:44.173331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f3a72628fd04c489680ab5cb90b4054', 'request_id': 'req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:44.187127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] The token cache is being invalidate because project 8f3a72628fd04c489680ab5cb90b4054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:44.187482 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:44.188108 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fcb2241-6b2a-468e-b005-4d98ba2f91ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:44.189907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1387/2805] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:44 2022] DELETE /identity/v3/projects/8f3a72628fd04c489680ab5cb90b4054 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:44.194859 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ec4e2-213f-4db0-8fc3-98a9f8cf4d4a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.195152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ec4e2-213f-4db0-8fc3-98a9f8cf4d4a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.195461 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ec4e2-213f-4db0-8fc3-98a9f8cf4d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.224107 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6f3ec4e2-213f-4db0-8fc3-98a9f8cf4d4a None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:44.225319 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6f3ec4e2-213f-4db0-8fc3-98a9f8cf4d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:44.273428 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:44.287537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1419/2806] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:44.296933 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd0b8a-5c30-4b35-94d7-337c27acb6af None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.297316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd0b8a-5c30-4b35-94d7-337c27acb6af None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.297662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd0b8a-5c30-4b35-94d7-337c27acb6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.318553 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-12bd0b8a-5c30-4b35-94d7-337c27acb6af None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:44.319785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-12bd0b8a-5c30-4b35-94d7-337c27acb6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:44.328304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1388/2807] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:44.332906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ec4e2-213f-4db0-8fc3-98a9f8cf4d4a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.343752 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ec4e2-213f-4db0-8fc3-98a9f8cf4d4a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.348387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555018b-c00e-484c-9736-402880670eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.348998 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555018b-c00e-484c-9736-402880670eed admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.349279 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555018b-c00e-484c-9736-402880670eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.349671 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555018b-c00e-484c-9736-402880670eed admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.350318 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555018b-c00e-484c-9736-402880670eed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:44.350684 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555018b-c00e-484c-9736-402880670eed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.352302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555018b-c00e-484c-9736-402880670eed admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.365606 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1420/2808] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:44.370890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd0b8a-5c30-4b35-94d7-337c27acb6af None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.380818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd0b8a-5c30-4b35-94d7-337c27acb6af None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.386455 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.387063 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.387425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.387661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.388271 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.390163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.420946 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1389/2809] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:44 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:44.425044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555018b-c00e-484c-9736-402880670eed admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.441955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555018b-c00e-484c-9736-402880670eed admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.523259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.523738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.524023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.524463 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.525127 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.527545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.559759 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1421/2810] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:44 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:44.564420 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.576183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd865c3-9c93-4143-9625-c48dd760ef8d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.580729 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.581338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.581743 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.582048 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.582704 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.584353 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.595049 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1390/2811] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:44.599251 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.609512 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255f323-46af-4963-baa6-157e3325b7bc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.614003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f8f3e-f073-436e-a2fb-3e4d37865883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.614601 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f8f3e-f073-436e-a2fb-3e4d37865883 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.614878 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f8f3e-f073-436e-a2fb-3e4d37865883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.615155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f8f3e-f073-436e-a2fb-3e4d37865883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.615832 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f8f3e-f073-436e-a2fb-3e4d37865883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.617549 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f8f3e-f073-436e-a2fb-3e4d37865883 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.627682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1422/2812] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:44.631960 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.642664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361474f5-6294-4f62-9ef8-df66f921c0a6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.647625 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.648213 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.648492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.648767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] PATH_INFO: `/v3/projects/18a5c184012b447688dd58a7316b4100/users/1d8014c0db084b4a9b9efb8155b62436/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.663832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a5c184012b447688dd58a7316b4100, user_id=1d8014c0db084b4a9b9efb8155b62436, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.667155 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.682201 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1391/2813] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:44 2022] PUT /identity/v3/projects/18a5c184012b447688dd58a7316b4100/users/1d8014c0db084b4a9b9efb8155b62436/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:44.692987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad73751-26f8-42e9-9608-b67e1517f82f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.693254 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad73751-26f8-42e9-9608-b67e1517f82f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.693537 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad73751-26f8-42e9-9608-b67e1517f82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.722186 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1ad73751-26f8-42e9-9608-b67e1517f82f None None] MFA Rules not processed for user `1d8014c0db084b4a9b9efb8155b62436`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:44.723478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1ad73751-26f8-42e9-9608-b67e1517f82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:44.802110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1423/2814] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:44.814481 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:44.816433 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:44.905751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a206b-e52f-461c-8325-8b2475feb9ba admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:44.910292 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9fffa-39e4-4fb5-a082-ac58d79ba68e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:44.910903 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9fffa-39e4-4fb5-a082-ac58d79ba68e service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:44.911205 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9fffa-39e4-4fb5-a082-ac58d79ba68e service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:44.911499 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9fffa-39e4-4fb5-a082-ac58d79ba68e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:44.925267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9fffa-39e4-4fb5-a082-ac58d79ba68e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:44.927507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9fffa-39e4-4fb5-a082-ac58d79ba68e service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:44.938425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1392/2815] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:49.644738 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad73751-26f8-42e9-9608-b67e1517f82f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:49.699322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:49.719502 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad73751-26f8-42e9-9608-b67e1517f82f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:49.725474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:49.726172 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:49.726451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:49.726725 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] PATH_INFO: `/v3/users/1d8014c0db084b4a9b9efb8155b62436` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:49.727973 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d8014c0db084b4a9b9efb8155b62436)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:49.729925 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:49.786202 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d8014c0db084b4a9b9efb8155b62436'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.786954 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d8014c0db084b4a9b9efb8155b62436'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.793907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d8014c0db084b4a9b9efb8155b62436'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.794542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d8014c0db084b4a9b9efb8155b62436'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.803538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d8014c0db084b4a9b9efb8155b62436'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:49.822395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1424/2816] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:49 2022] DELETE /identity/v3/users/1d8014c0db084b4a9b9efb8155b62436 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:49.826948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:49.827480 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:49.827581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:49.846911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:49.848169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:49.882849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:49.914512 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1393/2817] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:49.925465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:49.927157 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:49.989849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d9673-c986-4a5f-9d73-2ddb4193247c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:49.993767 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb11040-6b3c-4ddb-9e08-11460ae1138b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:49.994459 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11040-6b3c-4ddb-9e08-11460ae1138b service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:49.994741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11040-6b3c-4ddb-9e08-11460ae1138b service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:49.995018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11040-6b3c-4ddb-9e08-11460ae1138b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.005008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb11040-6b3c-4ddb-9e08-11460ae1138b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.006456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb11040-6b3c-4ddb-9e08-11460ae1138b service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.017380 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1425/2818] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:50.310458 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.312188 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:50.324592 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b5cec-89a2-45e0-8c3e-a244f8a622b5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.331736 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.332387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.332679 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.332977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] PATH_INFO: `/v3/projects/18a5c184012b447688dd58a7316b4100` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.334169 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a5c184012b447688dd58a7316b4100)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.336917 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.388402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a5c184012b447688dd58a7316b4100', 'request_id': 'req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.393831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a5c184012b447688dd58a7316b4100', 'request_id': 'req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.399955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a5c184012b447688dd58a7316b4100', 'request_id': 'req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.415018 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] The token cache is being invalidate because project 18a5c184012b447688dd58a7316b4100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:50.415331 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.415979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b2eb4419-6ec8-4336-a26d-b600d5eb60a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:50.417559 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1394/2819] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:50 2022] DELETE /identity/v3/projects/18a5c184012b447688dd58a7316b4100 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:50.422779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8710dcf-e380-4792-b404-67281889e0c3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.423068 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8710dcf-e380-4792-b404-67281889e0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.423369 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8710dcf-e380-4792-b404-67281889e0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.443866 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d8710dcf-e380-4792-b404-67281889e0c3 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:50.445070 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d8710dcf-e380-4792-b404-67281889e0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:50.508530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1426/2820] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:50.516656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4fdc-08c0-45d6-b439-128160cbd989 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.517013 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4fdc-08c0-45d6-b439-128160cbd989 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.517299 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4fdc-08c0-45d6-b439-128160cbd989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.540144 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-315f4fdc-08c0-45d6-b439-128160cbd989 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:50.541383 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-315f4fdc-08c0-45d6-b439-128160cbd989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:50.550720 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1395/2821] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:50.555366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8710dcf-e380-4792-b404-67281889e0c3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.566515 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8710dcf-e380-4792-b404-67281889e0c3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.572576 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.573181 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.573442 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.573710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.574368 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:50.574667 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.576481 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.583474 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:50.589708 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1427/2822] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:50.594354 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4fdc-08c0-45d6-b439-128160cbd989 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.603889 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4fdc-08c0-45d6-b439-128160cbd989 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.609564 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.610384 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.610661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.610953 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.611657 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.613584 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.647469 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1396/2823] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:50 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:50.652018 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.662599 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26097364-aa62-400c-b4da-0336fc21f7fb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.721605 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.722147 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.722429 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.722705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.723356 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.725286 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.747781 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1428/2824] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:50 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:50.752313 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.765474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de60a2-0059-4300-a459-3e5314bb81a1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.771229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.771833 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.772115 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.772387 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.773010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.774606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.785491 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1397/2825] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:50.791628 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.802479 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2724331-d451-4953-a6ae-a2147e85c058 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.807930 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08060a2-cef4-4df2-83a6-aa470fab3297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.808531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08060a2-cef4-4df2-83a6-aa470fab3297 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.808816 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08060a2-cef4-4df2-83a6-aa470fab3297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.809088 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08060a2-cef4-4df2-83a6-aa470fab3297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.809743 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08060a2-cef4-4df2-83a6-aa470fab3297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.811756 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08060a2-cef4-4df2-83a6-aa470fab3297 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.822300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1429/2826] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:50.826390 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:50.837154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bf7d2-50ff-47a6-b423-ece996c329a4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:50.842308 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:50.842943 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.843260 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.843548 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] PATH_INFO: `/v3/projects/3bed6d424303417caa1c190854e4f7ba/users/7c483e018faa452390ca08bc20f8e1c3/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.859366 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bed6d424303417caa1c190854e4f7ba, user_id=7c483e018faa452390ca08bc20f8e1c3, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:50.862035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:50.877070 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1398/2827] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:50 2022] PUT /identity/v3/projects/3bed6d424303417caa1c190854e4f7ba/users/7c483e018faa452390ca08bc20f8e1c3/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:50.888581 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:50.888844 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:50.889122 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:50.918003 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] MFA Rules not processed for user `7c483e018faa452390ca08bc20f8e1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:50.919616 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:50.990990 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1430/2828] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:51.005233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:51.007065 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:51.064996 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e11dbe-571c-4017-bbd9-e45bd42c7f71 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:51.068856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825c44f-eba7-4630-8365-164d0d9cdc73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:51.069460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825c44f-eba7-4630-8365-164d0d9cdc73 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:51.069742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825c44f-eba7-4630-8365-164d0d9cdc73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:51.070016 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825c44f-eba7-4630-8365-164d0d9cdc73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:51.078682 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825c44f-eba7-4630-8365-164d0d9cdc73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:51.080151 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825c44f-eba7-4630-8365-164d0d9cdc73 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:51.083683 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:51.089267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1399/2829] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:53.428611 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.430836 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:53.505656 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b27991-6804-4c3e-bd36-64312e0089a6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.510940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.511620 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.511905 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.512179 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] PATH_INFO: `/v3/users/cf0c764c2595424fba631d573f5b6883` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.513446 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0c764c2595424fba631d573f5b6883)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.515440 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.521690 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:53.568092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0c764c2595424fba631d573f5b6883'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.568799 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0c764c2595424fba631d573f5b6883'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.576315 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0c764c2595424fba631d573f5b6883'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.576987 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0c764c2595424fba631d573f5b6883'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.586037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0c764c2595424fba631d573f5b6883'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:53.602838 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1431/2830] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:53 2022] DELETE /identity/v3/users/cf0c764c2595424fba631d573f5b6883 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:53.607576 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.608064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.608163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.627652 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:53.628967 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:53.708337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1400/2831] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:53.720583 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:53.722415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:53.787227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2cb7-dda9-4bb4-a282-58ee95939972 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:53.790975 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2e8c0-9162-4509-9505-c30665e97cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:53.791621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2e8c0-9162-4509-9505-c30665e97cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:53.791906 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2e8c0-9162-4509-9505-c30665e97cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:53.792185 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2e8c0-9162-4509-9505-c30665e97cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:53.801720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2e8c0-9162-4509-9505-c30665e97cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:53.803166 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2e8c0-9162-4509-9505-c30665e97cf3 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:53.813711 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1432/2832] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:54.088558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.090712 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:54.110051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb4454-f12b-4c21-91ff-6762366ae661 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.116658 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.117389 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.117745 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.118164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] PATH_INFO: `/v3/projects/e9e96eecdb3d4498b0cc18846ebd5794` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.119708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e96eecdb3d4498b0cc18846ebd5794)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.122050 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.173763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e96eecdb3d4498b0cc18846ebd5794', 'request_id': 'req-149b909a-a016-4752-9a97-2a991392370c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:54.179810 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e96eecdb3d4498b0cc18846ebd5794', 'request_id': 'req-149b909a-a016-4752-9a97-2a991392370c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:54.187441 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e96eecdb3d4498b0cc18846ebd5794', 'request_id': 'req-149b909a-a016-4752-9a97-2a991392370c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:54.214181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] The token cache is being invalidate because project e9e96eecdb3d4498b0cc18846ebd5794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:54.214878 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:54.215714 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-149b909a-a016-4752-9a97-2a991392370c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:54.217567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1401/2833] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:54 2022] DELETE /identity/v3/projects/e9e96eecdb3d4498b0cc18846ebd5794 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:54.222199 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff7116-5a54-4710-bb1c-c2d67a8cc7ab None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.222554 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff7116-5a54-4710-bb1c-c2d67a8cc7ab None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.222834 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff7116-5a54-4710-bb1c-c2d67a8cc7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.243300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1dff7116-5a54-4710-bb1c-c2d67a8cc7ab None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:54.244584 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1dff7116-5a54-4710-bb1c-c2d67a8cc7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:54.314171 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1433/2834] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:54.322239 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666fa9-e9d8-4c20-860a-50a04d11a2b8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.322528 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666fa9-e9d8-4c20-860a-50a04d11a2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.322806 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666fa9-e9d8-4c20-860a-50a04d11a2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.343072 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0e666fa9-e9d8-4c20-860a-50a04d11a2b8 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:54.344304 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0e666fa9-e9d8-4c20-860a-50a04d11a2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:54.352460 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1402/2835] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:54.357152 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff7116-5a54-4710-bb1c-c2d67a8cc7ab None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.367869 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff7116-5a54-4710-bb1c-c2d67a8cc7ab None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.373907 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.374538 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.374824 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.375105 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.375803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:54.376110 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.377763 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.390828 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1434/2836] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:54.395692 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e666fa9-e9d8-4c20-860a-50a04d11a2b8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.405907 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e666fa9-e9d8-4c20-860a-50a04d11a2b8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.412107 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.412715 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.412974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.413249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.413894 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.415865 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.452540 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1403/2837] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:54.457338 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.468006 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8278-e574-4376-ab8d-3dfc2c2078c5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.480270 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:54.532661 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.533277 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.533561 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.533835 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.534665 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.536617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.559059 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1435/2838] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:54.564635 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.576318 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8518aa-76d3-493c-bd97-fddf1f38618b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.582085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.582716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.583009 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.583291 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.583951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.585593 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.596316 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1404/2839] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:54.600770 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.611550 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4265c-3adb-4a38-8edd-bdbfc9f982c0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.616691 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0aa96-2e4e-4c36-9c3f-73e34585bf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.617292 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0aa96-2e4e-4c36-9c3f-73e34585bf07 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.617589 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0aa96-2e4e-4c36-9c3f-73e34585bf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.617880 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0aa96-2e4e-4c36-9c3f-73e34585bf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.618528 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0aa96-2e4e-4c36-9c3f-73e34585bf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.620155 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0aa96-2e4e-4c36-9c3f-73e34585bf07 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.630741 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1436/2840] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:54.635534 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.651977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa2d57-fb62-4e21-8173-3533ab955188 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.660916 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.661572 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.661856 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.662132 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] PATH_INFO: `/v3/projects/d6647af6bb8843b4a78456ececc006f4/users/cf43c0e25ecf4e0885ff9589e21dc124/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.678585 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6647af6bb8843b4a78456ececc006f4, user_id=cf43c0e25ecf4e0885ff9589e21dc124, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.681325 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.701830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1405/2841] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:54 2022] PUT /identity/v3/projects/d6647af6bb8843b4a78456ececc006f4/users/cf43c0e25ecf4e0885ff9589e21dc124/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:54.777387 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.777679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.777949 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.807035 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] MFA Rules not processed for user `cf43c0e25ecf4e0885ff9589e21dc124`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:54.808303 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:54.876782 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1437/2842] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:51:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:54.888713 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:54.890905 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:54.915079 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:54.956027 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f1ec0-794f-433d-817c-cd09a52dadb5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:54.960609 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bd7fc-8bc2-4b31-93d7-d3d443cd0e9a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380786eb9b7848d59f3bb0007726dd2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:54.961275 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bd7fc-8bc2-4b31-93d7-d3d443cd0e9a service glance] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:54.961558 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bd7fc-8bc2-4b31-93d7-d3d443cd0e9a service glance] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:54.961832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bd7fc-8bc2-4b31-93d7-d3d443cd0e9a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:54.971614 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bd7fc-8bc2-4b31-93d7-d3d443cd0e9a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:54.973057 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bd7fc-8bc2-4b31-93d7-d3d443cd0e9a service glance] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:54.982337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1406/2843] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:01.403966 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:01.405797 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:01.479123 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdc774-7b03-472a-afdc-e99d24da5cd6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:01.486531 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:01.487300 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:01.488069 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:01.488445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] PATH_INFO: `/v3/users/7c483e018faa452390ca08bc20f8e1c3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:01.490094 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c483e018faa452390ca08bc20f8e1c3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:01.492098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:01.526164 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:01.554516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c483e018faa452390ca08bc20f8e1c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.555271 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c483e018faa452390ca08bc20f8e1c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.563025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c483e018faa452390ca08bc20f8e1c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.563812 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c483e018faa452390ca08bc20f8e1c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.573366 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c483e018faa452390ca08bc20f8e1c3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:01.592542 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1438/2844] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:01 2022] DELETE /identity/v3/users/7c483e018faa452390ca08bc20f8e1c3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:01.599742 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:01.600305 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:01.600415 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:01.620210 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:01.621425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:01.689346 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1407/2845] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:01.699545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:01.701282 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:01.776412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4e9b0-e4b0-4728-8ab9-a7491af42efe admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:01.781289 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad797739-2a38-4065-8bdf-3c300c3e8118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:01.781929 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad797739-2a38-4065-8bdf-3c300c3e8118 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:01.782210 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad797739-2a38-4065-8bdf-3c300c3e8118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:01.782585 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad797739-2a38-4065-8bdf-3c300c3e8118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:01.793222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad797739-2a38-4065-8bdf-3c300c3e8118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:01.794602 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad797739-2a38-4065-8bdf-3c300c3e8118 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:01.805198 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1439/2846] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:02.090884 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.092827 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.112980 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec7fb-9ee4-48a9-8295-8fe009c1ee88 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.119038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.120355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.120784 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.121100 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] PATH_INFO: `/v3/projects/3bed6d424303417caa1c190854e4f7ba` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.122737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bed6d424303417caa1c190854e4f7ba)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.125173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.182219 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bed6d424303417caa1c190854e4f7ba', 'request_id': 'req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.189768 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bed6d424303417caa1c190854e4f7ba', 'request_id': 'req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.196651 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bed6d424303417caa1c190854e4f7ba', 'request_id': 'req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.211804 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] The token cache is being invalidate because project 3bed6d424303417caa1c190854e4f7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:02.212095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.212939 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-6c5bceeb-0bb1-4be7-bf1a-7a0c9e253976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:02.215129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1408/2847] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:02 2022] DELETE /identity/v3/projects/3bed6d424303417caa1c190854e4f7ba => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:02.220188 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782e4b-8a23-4cba-afb1-c2fdeff2ea93 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.220779 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782e4b-8a23-4cba-afb1-c2fdeff2ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.221426 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782e4b-8a23-4cba-afb1-c2fdeff2ea93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.241710 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-85782e4b-8a23-4cba-afb1-c2fdeff2ea93 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:02.243044 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-85782e4b-8a23-4cba-afb1-c2fdeff2ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.306253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1440/2848] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:02.315445 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618af330-d8e7-46b3-aa88-12f259f0c47f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.315739 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618af330-d8e7-46b3-aa88-12f259f0c47f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.316019 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618af330-d8e7-46b3-aa88-12f259f0c47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.337628 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-618af330-d8e7-46b3-aa88-12f259f0c47f None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:02.338381 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-618af330-d8e7-46b3-aa88-12f259f0c47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.348454 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1409/2849] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:02.353201 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85782e4b-8a23-4cba-afb1-c2fdeff2ea93 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.364545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85782e4b-8a23-4cba-afb1-c2fdeff2ea93 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.370296 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.370904 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.371234 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.371533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.372192 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:02.372526 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.374170 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.387025 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1441/2850] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:02.392181 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618af330-d8e7-46b3-aa88-12f259f0c47f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.403157 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618af330-d8e7-46b3-aa88-12f259f0c47f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.408857 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.409492 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.409793 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.410106 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.410818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.412846 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.445648 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1410/2851] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:02 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:02.450190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.462009 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325856a-c5bc-4127-8906-0190b9c4a19e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.496919 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:02.533473 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.534015 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.534299 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.534617 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.535394 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.537621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.564348 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1442/2852] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:02 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:02.567940 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.579645 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64cae7-fe7c-4d6f-8506-629589d365d5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.585581 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.586203 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.586474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.586767 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.587444 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.589036 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.600187 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1411/2853] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:02.604402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.614953 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e0889-88ea-4b0a-a4ec-e39efa75113e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.619937 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c9704-4cd3-45e7-a46f-15041cbe8afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.620533 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c9704-4cd3-45e7-a46f-15041cbe8afd admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.620811 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c9704-4cd3-45e7-a46f-15041cbe8afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.621080 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c9704-4cd3-45e7-a46f-15041cbe8afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.621726 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c9704-4cd3-45e7-a46f-15041cbe8afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.623415 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c9704-4cd3-45e7-a46f-15041cbe8afd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.633377 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1443/2854] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:02.638148 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.644681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:02.649338 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d5fe1-575b-4025-9064-9d9112061a58 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.655448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.656142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.656428 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.656717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] PATH_INFO: `/v3/projects/df1bd62253b1499d967691558359d094/users/6e2a0358c66f47608554ef937ae57e50/roles/ffb20dd7d8254b1b960acfe117faa5c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.674849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1bd62253b1499d967691558359d094, user_id=6e2a0358c66f47608554ef937ae57e50, role_id=ffb20dd7d8254b1b960acfe117faa5c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.677474 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.693866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1412/2855] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:02 2022] PUT /identity/v3/projects/df1bd62253b1499d967691558359d094/users/6e2a0358c66f47608554ef937ae57e50/roles/ffb20dd7d8254b1b960acfe117faa5c3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:02.699434 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477757bf-81c4-4acf-984a-c3d2062de446 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.699728 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477757bf-81c4-4acf-984a-c3d2062de446 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.700004 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477757bf-81c4-4acf-984a-c3d2062de446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.719749 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-477757bf-81c4-4acf-984a-c3d2062de446 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:02.721341 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-477757bf-81c4-4acf-984a-c3d2062de446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.786372 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1444/2856] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:02.796134 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:02.797955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:02.870372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330aae57-2c62-41db-88d1-604a1cef10fe admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:02.870372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:02.870372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:02.870372 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:02.870754 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:02.877698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:02.879130 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:02.888434 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1413/2857] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:07.079031 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb5ec7-b095-43f8-b630-263994e2f0c5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.079644 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb5ec7-b095-43f8-b630-263994e2f0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.079874 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb5ec7-b095-43f8-b630-263994e2f0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.113390 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ddbb5ec7-b095-43f8-b630-263994e2f0c5 None None] MFA Rules not processed for user `6e2a0358c66f47608554ef937ae57e50`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:07.114758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ddbb5ec7-b095-43f8-b630-263994e2f0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:07.183532 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1445/2858] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:07.188674 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.199649 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817f68d-4006-4199-a9d6-35d54047d810 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.204693 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.205306 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.205716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.206143 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.206779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.208751 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.246061 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1414/2859] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:07 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:07.247892 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb5ec7-b095-43f8-b630-263994e2f0c5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.260047 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb5ec7-b095-43f8-b630-263994e2f0c5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.303959 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:07.323284 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.323940 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.324294 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.324633 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.325410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.327373 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.352466 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1446/2860] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:07 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:52:07.356733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.367849 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a03794-ff3d-4639-9973-81fd319fe11d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.373129 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.373803 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.374152 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.374436 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.375089 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.376759 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.388171 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1415/2861] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:07.392447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.403934 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1214f-f1ef-46d5-b9e1-b5f4341c595c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.409412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.409978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.410264 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.410543 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] PATH_INFO: `/v3/projects/98d4dbf837f7404a8ff6312b79a49b04/users/058c3b28e1df43f09e8a7a4ba4db37ea/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.427626 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d4dbf837f7404a8ff6312b79a49b04, user_id=058c3b28e1df43f09e8a7a4ba4db37ea, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.430314 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.445681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1447/2862] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:07 2022] PUT /identity/v3/projects/98d4dbf837f7404a8ff6312b79a49b04/users/058c3b28e1df43f09e8a7a4ba4db37ea/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:07.449763 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.460565 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4ea55-3a0e-4899-bc17-00f45845cd8b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.519448 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4bcb7-8e13-4615-a2a9-dc3346bace64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.520059 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4bcb7-8e13-4615-a2a9-dc3346bace64 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.520343 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4bcb7-8e13-4615-a2a9-dc3346bace64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.520619 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4bcb7-8e13-4615-a2a9-dc3346bace64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.521310 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4bcb7-8e13-4615-a2a9-dc3346bace64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.522927 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4bcb7-8e13-4615-a2a9-dc3346bace64 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.532911 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1416/2863] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:52:07.537516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:07.548273 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56f4d-57f5-482c-a5fb-a95a0c65b461 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:07.553224 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:07.553918 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:07.554207 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:07.554487 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] PATH_INFO: `/v3/projects/98d4dbf837f7404a8ff6312b79a49b04/users/058c3b28e1df43f09e8a7a4ba4db37ea/roles/c64ac97b1795434390dc18ab162aec39` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:07.556516 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d4dbf837f7404a8ff6312b79a49b04, user_id=058c3b28e1df43f09e8a7a4ba4db37ea, role_id=c64ac97b1795434390dc18ab162aec39)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:07.559176 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:07.570976 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1448/2864] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:07 2022] PUT /identity/v3/projects/98d4dbf837f7404a8ff6312b79a49b04/users/058c3b28e1df43f09e8a7a4ba4db37ea/roles/c64ac97b1795434390dc18ab162aec39 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:12.446173 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a53eba-372f-4c52-9e92-2d0cebd48338 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:12.446698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a53eba-372f-4c52-9e92-2d0cebd48338 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:12.446991 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a53eba-372f-4c52-9e92-2d0cebd48338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:12.489594 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a0a53eba-372f-4c52-9e92-2d0cebd48338 None None] MFA Rules not processed for user `058c3b28e1df43f09e8a7a4ba4db37ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:12.491064 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a0a53eba-372f-4c52-9e92-2d0cebd48338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:12.547391 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:12.605218 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1417/2865] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:12.620445 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:12.623302 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:12.693033 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598e9e-2f6a-482e-99ec-4c890b039bbb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:12.697734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '380786eb9b7848d59f3bb0007726dd2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:12.698514 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:12.699174 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:12.699604 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:12.767849 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:12.769305 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:12.778488 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1449/2866] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:12.918644 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:12.918966 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:12.919249 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:12.954716 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:12.956718 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:13.026832 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1418/2867] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:52:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:13.076865 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:13.078447 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:13.132637 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:13.154456 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834959-dd85-4b4d-bc16-4ed097508513 service glance] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:13.160669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26f1f1-7075-4347-885b-f89fad29b6b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:13.161276 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f1f1-7075-4347-885b-f89fad29b6b0 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:13.161555 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f1f1-7075-4347-885b-f89fad29b6b0 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:13.161825 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f1f1-7075-4347-885b-f89fad29b6b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:13.170529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26f1f1-7075-4347-885b-f89fad29b6b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:13.172163 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26f1f1-7075-4347-885b-f89fad29b6b0 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:13.181227 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1450/2868] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:52:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:52:20.516494 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.521337 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.595977 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0904d-ec74-4819-97f5-4271aaf15d2a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.601800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.602555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.602893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.603163 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] PATH_INFO: `/v3/users/cf43c0e25ecf4e0885ff9589e21dc124` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.604439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf43c0e25ecf4e0885ff9589e21dc124)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.608332 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.673231 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf43c0e25ecf4e0885ff9589e21dc124'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.682555 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf43c0e25ecf4e0885ff9589e21dc124'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.692948 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf43c0e25ecf4e0885ff9589e21dc124'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.693681 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf43c0e25ecf4e0885ff9589e21dc124'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.694369 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf43c0e25ecf4e0885ff9589e21dc124'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.695942 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:20.716831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1419/2869] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:20 2022] DELETE /identity/v3/users/cf43c0e25ecf4e0885ff9589e21dc124 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:20.722158 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.723095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.723095 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.745860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] MFA Rules not processed for user `66932a6f9a244fcaa7375a468f28c399`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:20.747211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.825149 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1451/2870] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:20.832233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.833866 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.910602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee24af-0764-4d44-89ab-64b1a272e38e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.914323 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a58d0-1a2f-423e-98bf-1a0c196bd914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.914955 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a58d0-1a2f-423e-98bf-1a0c196bd914 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.915259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a58d0-1a2f-423e-98bf-1a0c196bd914 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.915602 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a58d0-1a2f-423e-98bf-1a0c196bd914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.927864 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a58d0-1a2f-423e-98bf-1a0c196bd914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.931236 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a58d0-1a2f-423e-98bf-1a0c196bd914 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.944004 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1420/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:21.276311 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.277920 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:21.293112 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7e7e8-70d7-482d-8ee2-989a8e0209e5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.308829 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.309632 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.310342 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.312195 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] PATH_INFO: `/v3/projects/d6647af6bb8843b4a78456ececc006f4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.316382 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6647af6bb8843b4a78456ececc006f4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.322037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.404383 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6647af6bb8843b4a78456ececc006f4', 'request_id': 'req-a4530bf1-3d7a-4308-a66e-f103ff4069ae'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.414709 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6647af6bb8843b4a78456ececc006f4', 'request_id': 'req-a4530bf1-3d7a-4308-a66e-f103ff4069ae'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.420545 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6647af6bb8843b4a78456ececc006f4', 'request_id': 'req-a4530bf1-3d7a-4308-a66e-f103ff4069ae'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.439621 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] The token cache is being invalidate because project d6647af6bb8843b4a78456ececc006f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:21.440098 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.440860 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:21.442508 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1452/2872] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:21 2022] DELETE /identity/v3/projects/d6647af6bb8843b4a78456ececc006f4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:21.980085 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aabca2-29ae-471a-ad6b-3bc2840667cf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.980633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aabca2-29ae-471a-ad6b-3bc2840667cf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.980633 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aabca2-29ae-471a-ad6b-3bc2840667cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.001852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-06aabca2-29ae-471a-ad6b-3bc2840667cf None None] MFA Rules not processed for user `6e2a0358c66f47608554ef937ae57e50`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:22.003349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-06aabca2-29ae-471a-ad6b-3bc2840667cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:22.066233 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1421/2873] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:52:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:23.163437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.165045 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:23.241436 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:23.271572 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4530bf1-3d7a-4308-a66e-f103ff4069ae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.277587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.278194 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.278472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.278736 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.280679 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:23.293203 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.294792 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.304685 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1453/2874] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:52:24.117349 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:24.117934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:24.117934 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:24.137988 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:24.139248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:24.197711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1422/2875] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Aug 24 06:52:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:24.210801 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:24.212861 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:24.287412 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc869361-f31d-4584-be53-d03eb7a06216 service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:24.293077 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be24d7a-350f-4238-990d-fcea38785681 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:24.293773 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be24d7a-350f-4238-990d-fcea38785681 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:24.294052 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be24d7a-350f-4238-990d-fcea38785681 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:24.294329 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be24d7a-350f-4238-990d-fcea38785681 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:24.303037 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be24d7a-350f-4238-990d-fcea38785681 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:24.304666 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be24d7a-350f-4238-990d-fcea38785681 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:24.313138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1454/2876] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:52:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:52:34.190880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=83,keystone:DELETE=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:34.310327 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=45 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:51.161024 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:51.162867 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:51.227012 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d821d-de44-463b-bae2-05a739f1838b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:51.231451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:51.232074 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:51.232355 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:51.232632 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:51.234800 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:51.298814 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:51.300265 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:51.310533 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1423/2877] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:01.306031 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=50 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:29.536336 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e25ee-8878-4234-a712-cae07305d62e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.536867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e25ee-8878-4234-a712-cae07305d62e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.536867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e25ee-8878-4234-a712-cae07305d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.556566 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-931e25ee-8878-4234-a712-cae07305d62e None None] MFA Rules not processed for user `0b23553f7543446cb5eca18f26e25816`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:29.558258 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-931e25ee-8878-4234-a712-cae07305d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:29.615750 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1455/2878] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:53:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:29.653711 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.655247 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:29.721737 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610f504-62ed-49a9-a6cf-b199ef7a6ded service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.725821 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.726450 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.726734 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.727010 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.729217 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:29.828248 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.829698 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.850777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1424/2879] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:30.645072 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e25ee-8878-4234-a712-cae07305d62e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:30.657431 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e25ee-8878-4234-a712-cae07305d62e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:30.661092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:30.661714 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:30.662124 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:30.662402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:30.664313 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:30.682955 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:30.684464 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:30.697867 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1456/2880] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:39.848105 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=63 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:40.692204 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=31 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:45.572127 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:45.573899 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:45.586311 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6e3a6-a413-48ed-b2f4-0bb8a20e9fe9 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:45.590195 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcc238-25b9-4e20-98d3-a763ef83e771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:45.590809 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcc238-25b9-4e20-98d3-a763ef83e771 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:45.591095 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcc238-25b9-4e20-98d3-a763ef83e771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:45.591402 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcc238-25b9-4e20-98d3-a763ef83e771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:45.594000 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d8bcc238-25b9-4e20-98d3-a763ef83e771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:45.604656 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcc238-25b9-4e20-98d3-a763ef83e771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:45.606093 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcc238-25b9-4e20-98d3-a763ef83e771 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:45.614688 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1425/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:55.611083 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:15.572627 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:15.575817 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:15.593450 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e702d1-0619-40d7-a85c-5503ad45c4ff service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:15.601243 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:15.602062 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:15.602413 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:15.602803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:15.604713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:15.618082 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:15.619771 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:15.632734 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1457/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:17.050545 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:17.051045 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:17.051276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:17.104486 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] MFA Rules not processed for user `f2837a414b814cf7915592c7f67cbd01`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:17.105733 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:17.123951 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1426/2883] 10.222.0.52 () {58 vars in 967 bytes} [Wed Aug 24 06:54:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:17.164735 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:17.188211 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2044a-345a-4290-b2d3-7edacac09e5e service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:17.194465 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ca3818552143069b0bacbb5bec7b62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:17.195078 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:17.195402 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:17.195681 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:17.204971 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:17.206587 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:17.215680 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1458/2884] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Aug 24 06:54:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:54:23.992893 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:23.994735 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:24.069071 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddb385-a0b3-4615-8554-9b07a756e631 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:24.074199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:24.074971 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:24.075285 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:24.075569 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] PATH_INFO: `/v3/users/6e2a0358c66f47608554ef937ae57e50` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:24.076785 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2a0358c66f47608554ef937ae57e50)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:24.078738 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:24.127425 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2a0358c66f47608554ef937ae57e50'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.135302 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2a0358c66f47608554ef937ae57e50'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.143813 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2a0358c66f47608554ef937ae57e50'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.144427 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2a0358c66f47608554ef937ae57e50'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.145179 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2a0358c66f47608554ef937ae57e50'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.165177 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1427/2885] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:54:23 2022] DELETE /identity/v3/users/6e2a0358c66f47608554ef937ae57e50 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:24.354143 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:24.366683 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c094f9d-cc15-4d2b-8ef8-b15438e9acab service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:24.439102 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:24.439777 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:24.440058 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:24.440385 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] PATH_INFO: `/v3/projects/df1bd62253b1499d967691558359d094` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:24.442818 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df1bd62253b1499d967691558359d094)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:24.447151 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:24.497138 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df1bd62253b1499d967691558359d094', 'request_id': 'req-7e739f17-cde4-400a-9eef-086fbb88cc40'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.503578 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df1bd62253b1499d967691558359d094', 'request_id': 'req-7e739f17-cde4-400a-9eef-086fbb88cc40'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.509586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df1bd62253b1499d967691558359d094', 'request_id': 'req-7e739f17-cde4-400a-9eef-086fbb88cc40'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.525190 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] The token cache is being invalidate because project df1bd62253b1499d967691558359d094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:54:24.525491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.526126 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.527688 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1459/2886] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:54:24 2022] DELETE /identity/v3/projects/df1bd62253b1499d967691558359d094 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:24.531880 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:24.533345 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:24.546606 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0dfb6-c8a0-407a-a96e-2b9ace050262 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:24.585502 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:24.605852 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:24.606488 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:24.606765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:24.607035 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] PATH_INFO: `/v3/users/058c3b28e1df43f09e8a7a4ba4db37ea` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:24.608196 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=058c3b28e1df43f09e8a7a4ba4db37ea)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:24.610142 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:24.660801 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058c3b28e1df43f09e8a7a4ba4db37ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.667519 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058c3b28e1df43f09e8a7a4ba4db37ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.676170 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058c3b28e1df43f09e8a7a4ba4db37ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.676841 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058c3b28e1df43f09e8a7a4ba4db37ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.677505 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f32222b8-d492-42ff-bfd0-9029882b0b5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058c3b28e1df43f09e8a7a4ba4db37ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:24.698199 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1428/2887] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:54:24 2022] DELETE /identity/v3/users/058c3b28e1df43f09e8a7a4ba4db37ea => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:24.883886 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:24.885478 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:24.898682 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e739f17-cde4-400a-9eef-086fbb88cc40 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:24.953702 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:24.961917 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:24.962586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:24.962877 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:24.963472 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] PATH_INFO: `/v3/projects/98d4dbf837f7404a8ff6312b79a49b04` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:24.964978 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d4dbf837f7404a8ff6312b79a49b04)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:24.967008 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:25.020510 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d4dbf837f7404a8ff6312b79a49b04', 'request_id': 'req-afde676b-d856-4289-806b-82ac0c53c8d6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:25.027023 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d4dbf837f7404a8ff6312b79a49b04', 'request_id': 'req-afde676b-d856-4289-806b-82ac0c53c8d6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:25.032758 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d4dbf837f7404a8ff6312b79a49b04', 'request_id': 'req-afde676b-d856-4289-806b-82ac0c53c8d6'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:25.046946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] The token cache is being invalidate because project 98d4dbf837f7404a8ff6312b79a49b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:54:25.047392 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:25.048024 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:25.049530 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1460/2888] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:54:24 2022] DELETE /identity/v3/projects/98d4dbf837f7404a8ff6312b79a49b04 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:34.692926 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:35.043911 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:35.173850 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:35.174466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:35.174466 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:35.194664 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] MFA Rules not processed for user `0b23553f7543446cb5eca18f26e25816`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:55:35.195928 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:35.257567 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1429/2889] 10.222.0.52 () {58 vars in 958 bytes} [Wed Aug 24 06:55:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:55:36.221845 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:36.223618 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:36.285063 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde676b-d856-4289-806b-82ac0c53c8d6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:36.290568 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c216522efef4447caac245392d95b985', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:36.291231 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:36.291524 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:36.291802 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:36.294048 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:36.306169 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:36.307674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:36.319090 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1461/2890] 10.222.0.52 () {60 vars in 1396 bytes} [Wed Aug 24 06:55:36 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:55:36.487395 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:36.488855 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:36.584003 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111a140-21ac-407b-8487-1498cdb53b07 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:36.591518 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:36.592165 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:36.592503 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:36.592779 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:36.595164 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:36.608220 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:36.609825 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:36.620717 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1430/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:46.313398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=26 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:46.616284 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=57 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:49.997586 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:49.999092 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:50.076395 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda43aae-48e4-46f6-bcc1-7b83e4fc73d5 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:50.081995 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e1ec869263246f58f5a32d855a3d302', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:50.082614 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:50.082899 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:50.083177 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:50.085011 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:50.098093 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:50.099764 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:50.110914 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1462/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:00.106720 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=34 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:29.933702 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:29.935365 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:29.950765 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e32b2e-9677-4f71-9e11-84b04dab1a29 service placement] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:29.956577 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:29.957207 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:29.957542 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:29.958154 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:29.959904 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:29.979139 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:29.981439 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:29.992890 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1431/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:57:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:57:39.832669 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:39.834137 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:39.891598 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4f136-95fe-410c-9875-bda5876c6bfe service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:39.895491 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc4592ef54b41059fa7767a5ca3aaa3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:39.896128 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:39.896410 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:39.896686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:39.898386 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:39.963468 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:39.964961 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:39.975490 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1463/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:57:39.987830 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:43.138580 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:43.151259 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896cf83-3c7f-484d-a845-3e7e20079cc7 service placement] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:43.157030 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:43.157797 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:43.158963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:43.158963 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:43.165777 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:43.181116 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:43.184578 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:43.202896 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1432/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 06:57:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:57:46.258851 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:46.260400 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:46.273259 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f956c-6fe6-4ef0-a3e8-38875d90973e service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:46.278067 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:46.278705 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:46.278981 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:46.279407 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] PATH_INFO: `/v3/users/0b23553f7543446cb5eca18f26e25816` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:46.280401 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b23553f7543446cb5eca18f26e25816)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:46.282312 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:46.330946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b23553f7543446cb5eca18f26e25816'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.331675 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b23553f7543446cb5eca18f26e25816'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.338762 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b23553f7543446cb5eca18f26e25816'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.339451 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b23553f7543446cb5eca18f26e25816'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.348153 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b23553f7543446cb5eca18f26e25816'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.367040 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1464/2896] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:57:46 2022] DELETE /identity/v3/users/0b23553f7543446cb5eca18f26e25816 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:57:46.554447 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:46.566276 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7f0c-fa11-40a0-95e6-c849f3c7b2ef service placement] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:46.637409 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:46.638038 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:46.638328 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:46.638627 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] PATH_INFO: `/v3/projects/13dae75bec47489cac0248f2bb2c197d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:46.639973 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13dae75bec47489cac0248f2bb2c197d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:46.642286 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:46.692979 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13dae75bec47489cac0248f2bb2c197d', 'request_id': 'req-2ed1a894-4267-4f1d-a8b2-4b370272968a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.698422 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13dae75bec47489cac0248f2bb2c197d', 'request_id': 'req-2ed1a894-4267-4f1d-a8b2-4b370272968a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.705168 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13dae75bec47489cac0248f2bb2c197d', 'request_id': 'req-2ed1a894-4267-4f1d-a8b2-4b370272968a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.722818 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] The token cache is being invalidate because project 13dae75bec47489cac0248f2bb2c197d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:57:46.723229 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.724183 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.726507 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1433/2897] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:57:46 2022] DELETE /identity/v3/projects/13dae75bec47489cac0248f2bb2c197d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:57:46.730931 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:46.732529 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:46.746706 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba2eaa-508b-4e8b-94e3-f18324c5d0bc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:46.765352 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:46.808398 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:46.809028 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:46.809322 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:46.809595 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] PATH_INFO: `/v3/users/6d2c1bdf17fd485fb59273543850f2eb` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:46.810713 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2c1bdf17fd485fb59273543850f2eb)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:46.812686 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:46.860454 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2c1bdf17fd485fb59273543850f2eb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.861253 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2c1bdf17fd485fb59273543850f2eb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.867744 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2c1bdf17fd485fb59273543850f2eb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.868375 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2c1bdf17fd485fb59273543850f2eb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.876831 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2c1bdf17fd485fb59273543850f2eb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:46.901803 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1465/2898] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Aug 24 06:57:46 2022] DELETE /identity/v3/users/6d2c1bdf17fd485fb59273543850f2eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:57:47.086902 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:47.088661 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:47.102267 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1a894-4267-4f1d-a8b2-4b370272968a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:47.158708 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66932a6f9a244fcaa7375a468f28c399', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d0ff1eb049e4a50aede95b2f05e94b7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:47.159379 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:47.159662 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:47.159935 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] PATH_INFO: `/v3/projects/9bc1ce04283647c5a0693deee538d90d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:47.161051 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc1ce04283647c5a0693deee538d90d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:47.162974 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:47.184225 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:47.212208 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc1ce04283647c5a0693deee538d90d', 'request_id': 'req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:47.217829 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc1ce04283647c5a0693deee538d90d', 'request_id': 'req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:47.225789 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc1ce04283647c5a0693deee538d90d', 'request_id': 'req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:47.243521 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] The token cache is being invalidate because project 9bc1ce04283647c5a0693deee538d90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:57:47.243831 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:47.244451 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c1d92ddd-d093-478a-8a9f-fa9a8775b56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:47.245957 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1434/2899] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Aug 24 06:57:47 2022] DELETE /identity/v3/projects/9bc1ce04283647c5a0693deee538d90d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:57:56.897896 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:57.240411 dv-cinder-854271-1-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:02:38.143946 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:02:38.145573 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:38.290437 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25d486-7903-4002-9b8a-fa457178590c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:02:38.351588 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127021-438b-4b71-94f0-b421f8e97708 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd115d3f3adc54a17bf80962a6ae3ab82', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f23638950c1948f2ae7a7b13c645cd54', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:02:38.352222 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127021-438b-4b71-94f0-b421f8e97708 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:02:38.352501 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127021-438b-4b71-94f0-b421f8e97708 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:02:38.352768 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127021-438b-4b71-94f0-b421f8e97708 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:02:38.354674 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b3127021-438b-4b71-94f0-b421f8e97708 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:02:38.366051 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127021-438b-4b71-94f0-b421f8e97708 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:02:38.367659 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127021-438b-4b71-94f0-b421f8e97708 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:02:38.377544 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1466/2900] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Aug 24 07:02:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:02:48.373774 dv-cinder-854271-1-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}